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ABSTRACT 


The  professional  development  of  teachers  is  a crucial 
element  of  the  nation's  efforts  to  improve  education.  In  recent  years,  these 
efforts  have  sought  to  foster  high  standards  for  teaching  and  learning  for 
all  of  the  nation's  children,  and  almost  all  states  have  met  federal 
requirements  for  developing  challenging  statewide  content  standards.  Such 
standards  seek  a fundamental  shift  in  what  students  learn.  However, 
children's  learning  will  be  transformed  only  if  high  standards  are  reflected 
in  teachers'  classroom  practice.  Education  reforms  will  not  succeed  without 
teachers  who  are  immersed  in  the  subjects  they  teach  and  who  know  how  to 
foster  both  basic  knowledge  and  advanced  thinking  and  problem  solving  among 
their  students.  The  Eisenhower  Professional  Development  Program,  Title  II  of 
the  Elementary  and  Secondary  Education  Act  (ESEA) , is  the  federal 
government's  largest  investment  that  is  solely  focused  on  developing  the 
knowledge  and  skills  of  classroom  teachers.  This  is  the  second  report  of  the 
National  Evaluation  of  the  Eisenhower  Professional  Development  Program,  Part 
B (State  and  Local  Activities) , a multi-year  evaluation  being  conducted  by 
the  American  Institutes  for  Research  (AIR)  under  a contract  with  the  U.S. 
Department  of  Education's  Planning  and  Evaluation  Service.  The  first  report 
was  based  on  six  exploratory  case  studies  of  school  districts  conducted 
during  the  first  months  of  the  evaluation.  This  second  report  describes  the 
current  status  of  the  program,  based  primarily  on  data  from  national 
probability  samples  of  districts,  SAHE  grantees  (i.e.,  the  institutions  of 
higher  education  and  nonprofit  organizations  supported  through  the  SAHE 
component  of  the  program) , and  teachers,  as  well  as  on  data  from  10  in-depth 
case  studies  in  5 states.  Contains  140  references.  (Author/WRM) 


Reproductions  supplied  by  EDRS  are  the  best  that  can  be  made 
from  the  original  document. 
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EXECUTIVE  SUMMARY 

DESIGNING  EFFECTIVE  PROFESSIONAL  DEVELOPMENT: 
LESSONS  FROM  THE  EISENHOWER  PROGRAM 


The  professional  development  of  teachers  is  a crucial  element  of  the  nation’s  efforts  to 
improve  education.  In  recent  years,  these  efforts  have  sought  to  foster  high  standards  for  teaching 
and  learning  for  all  of  the  nation’s  children,  and  almost  all  states  have  met  federal  requirements  for 
developing  challenging  statewide  content  standards.  Such  standards  seek  a fundamental  shift  in  what 
students  learn.  However,  children’s  learning  will  be  transformed  only  if  high  standards  are  reflected 
in  teachers’  classroom  practice.  Education  reforms  will  not  succeed  without  teachers  who  are 
immersed  in  the  subjects  they  teach  and  who  know  how  to  foster  both  basic  knowledge  and  advanced 
thinking  and  problem  solving  among  their  students. 

The  Program:  The  Eisenhower  Professional  Development  Program,  Title  II  of  the 
Elementary  and  Secondary  Education  Act  (ESEA),  is  the  federal  government’s  largest  investment 
that  is  solely  focused  on  developing  the  knowledge  and  skills  of  classroom  teachers.  The  program  is 
key  to  meeting  the  U.S.  Department  of  Education’s  objective  of  ensuring  that  a “talented  and 
dedicated  teacher  is  in  every  classroom  in  America”.1  Part  B of  the  Eisenhower  Professional 
Development  Program,  with  a 1999  appropriation  of  about  $335  million,  provides  funds  through 
state  education  agencies  (SEAs)  to  school  districts,  and  through  state  agencies  for  higher  education 
(SAHEs)  to  institutions  of  higher  education  and  nonprofit  organizations.  These  funds  primarily 
support  professional  development  in  mathematics  and  science. 

The  Evaluation  Report:  This  is  the  second  report  of  the  National  Evaluation  of  the 
Eisenhower  Professional  Development  Program,  Part  B (State  and  Local  Activities),  a multi-year 
evaluation  being  conducted  by  the  American  Institutes  for  Research  (AIR)  under  a contract  with  the 
U.S.  Department  of  Education’s  Planning  and  Evaluation  Service.  The  first  report  was  based  on  six 
exploratory  case  studies  of  school  districts  conducted  during  the  first  months  of  the  evaluation.2  The 
purpose  of  that  report  was  to  obtain  an  initial  description  of  the  program  and  the  issues  that  it  faced  in 
different  local  contexts.  This  second  report  describes  the  current  status  of  the  program,  based 
primarily  on  data  from  national  probability  samples  of  districts,  SAHE  grantees  (i.e.,  the  institutions 
of  higher  education  and  nonprofit  organizations  supported  through  the  SAHE  component  of  the 
program),  and  teachers,  as  well  as  on  data  from  10  in-depth  case  studies  in  five  states.  The  third 
report  of  the  evaluation,  to  be  issued  in  early  2000,  will  augment  the  results  reported  here  with  a 
longitudinal  account  of  teachers'  experiences  in  Eisenhower  and  other  professional  development 
activities  and  resulting  changes  in  their  teaching  practices. 


The  Policy  Context 

In  response  to  public  concerns  about  education,  state  and  local  governments  have  taken  steps 
to  increase  children’s  achievement  in  school.  Many  states  and  school  districts  have  adopted  rigorous 
content  standards,  as  well  as  student  performance  standards,  which  describe  the  breadth  and  depth  at 
which  students  should  master  content.3  The  federal  government,  too,  has  moved  to  support  states  in 
their  development  of  content  and  student  performance  standards. 
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National,  state,  and  local  efforts  to  improve  education  are  intended  to  create  a fundamental 
shift  in  what  students  learn  and  how  they  are  taught.  The  success  of  such  ambitious  education  reform 
initiatives  hinges,  in  large  part,  on  the  qualifications  of  teachers.  However,  while  teachers  generally 
support  high  standards  for  teaching  and  learning,  many  teachers  are  not  prepared  to  implement 
teaching  practices  based  on  high  standards.4  Many  teachers  learned  to  teach  using  a model  of 
teaching  and  learning  that  focuses  heavily  on  memorizing  facts,  without  also  emphasizing  deeper 
understanding  of  subject  matter.5 

As  a result,  teacher  professional  development  is  a major  focus  of  systemic  reform  initiatives.6 
The  need  for  high-quality  professional  development  that  focuses  on  subject-matter  content  and  how 
students  learn  that  content  is  all  the  more  pressing  in  light  of  the  many  teachers  who  teach  outside  of 
their  areas  of  specialization.7  In  1998,  12  percent  of  science  teachers  of  students  in  grades  7-12,  and 
1 8 percent  of  mathematics  teachers  in  these  grades,  had  neither  a major  nor  a minor  in  their  main 
teaching  assignment.8  This  situation  is  especially  true  of  teachers  who  teach  at-risk  students  and 
those  who  teach  in  high-poverty  schools.  In  1998,  teachers  lacking  a major  in  their  primary 
assignment  taught  almost  a quarter  of  the  classes  in  high-poverty  schools,  compared  to  14  percent  of 
classes  in  low-poverty  schools.9 

The  Eisenhower  Professional  Development  Program,  established  in  1984  and  reauthorized  in 
1988  and  1994,  aims  to  support  high-quality  professional  development  to  help  teachers  meet  the 
demands  of  teaching  to  high  standards.  Yet,  the  1988-89  evaluation  of  Eisenhower  indicated  that 
district-supported  activities,  which  account  for  the  vast  majority  of  program  funds,  tended  to  be  one- 
time in-service  training  events,  averaging  six  hours  in  length.10  The  1994  reauthorization  intended  to 
shift  program-funded  activities  away  from  short-term  professional  development  toward  longer,  more 
intensive  activities. 


The  Eisenhower  Program 

The  Eisenhower  program  was  established  in  1984,  and  reauthorized  as  Title  II  of  the 
Elementary  and  Secondary  Education  Act,  as  amended  by  the  Improving  America’s  Schools  Act 
(IAS  A)  of  1994.  The  program  allocates  funds  through  states  to  school  districts  and  to  institutions  of 
higher  education  or  nonprofit  organizations.  In  fiscal  year  1999,  $335  million  was  appropriated  for 
Part  B of  the  program,  State  and  Local  Activities.  Eisenhower  funds  are  distributed  to  states 
according  to  a formula  that  weights  equally  the  number  of  children  in  the  state  between  the  ages  of  5 
and  17  and  the  state’s  allocation  under  Title  I,  Part  A of  the  Elementary  and  Secondary  Education 
Act.1 112  Eighty-four  percent  of  allocated  Title  II,  Part  B funds  are  distributed  to  SEAs,  with  the 
remaining  16  percent  allocated  to  SAHEs.  At  least  90  percent  of  SEA  allocations  then  flow  through 
to  local  education  agencies  (LEAs),  based  on  the  same  formula  (equal  weights  to  the  school-aged 
population  and  the  LEA’s  Title  I,  Part  A allocation). 13  LEAs  that  receive  Eisenhower  grants  under 
$10,000  are  required  to  form  consortia  with  other  such  LEAs,  unless  the  SEA  waives  the  requirement 
(Section  2204(b)).  SAHEs  distribute  at  least  95  percent  of  their  Eisenhower  allocations  by 
competitive  grants  or  contracts  to  institutions  of  higher  education  (IHEs)  or  nonprofit  organizations 
(NPOs)  that  provide  professional  development  to  teachers  or  prospective  teachers.  Each  SAHE 
develops  priorities  and  guidelines  for  the  awards  based  on  the  state  plan  for  improvement  in  teaching 
and  learning,  which  it  develops  collaboratively  with  the  SEA  (Section  2205(a)(2)(A)). 

Like  its  predecessor,  the  Eisenhower  Mathematics  and  Science  Education  Program,  the 
reauthorized  Eisenhower  program  focuses  on  the  professional  development  of  mathematics  and 
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science  teachers.  The  reauthorized  legislation,  however,  allows  states  and  districts  to  use  funds  in 
excess  of  $250  million  to  provide  professional  development  to  teachers  in  other  core  academic 
subject  areas  (Section  2206).14  In  its  1994  reauthorization  of  the  program,  Congress  made  it  explicit 
that  Eisenhower-assisted  activities  should  be  designed  to  improve  teacher  practice,  especially  for 
students  who  are  most  at  risk  of  school  failure.15 

■ The  Eisenhower  program  is  designed  to  support  high-quality  professional 
development  activities.  Both  the  Eisenhower  legislation  and  the  program  guidance 
published  by  the  Department  of  Education  (ED)  emphasize  that  the  Eisenhower  program 
should  support  high-quality  professional  development  activities— professional 
development  that  is  sustained,  intensive,  and  continuing.  According  to  the  law,  such 
professional  development  should  reflect  recent  research  on  teaching  and  learning  and 
should  provide  teachers  and  other  school  staff  with  the  knowledge  and  skills  necessary  to 
provide  all  students  with  the  opportunity  to  meet  challenging  standards. 

■ The  Eisenhower  legislation  encourages  the  use  of  funds  to  target  teachers  of  at-risk 
students.  The  1994  legislation  requires  that  state  applications  and  local  plans  take  into 
account  the  educational  needs  of  students  from  historically  underrepresented  populations. 
The  Eisenhower  legislation  places  special  emphasis  on  addressing  the  needs  of  teachers 
in  schools  receiving  Title  I,  Part  A funds;  generally  these  are  schools  that  have  higher 
rates  of  poverty  than  other  schools  in  their  districts. 

The  reauthorized  program  includes  a number  of  procedures  designed  to  ensure  the  provision 
of  high-quality  professional  development,  including  alignment  with  district  and  state  standards  and 
assessments,  coordination  (co-funding  and  working  with  schools  and  professional  development 
providers),  continuous  improvement  (use  of  indicators  and  needs  assessments),  and  teacher 
participation  in  planning. 

■ Districts  and  SAHE  grantees  are  required  to  align  their  Eisenhower-assisted 
activities  with  state  and  local  standards  and  reforms.  Recent  efforts  to  improve 
education  have  focused  on  ensuring  that  all  aspects  of  the  education  system — including 
curricula,  assessments,  teacher  education — be  consistent  with  one  another  and  be  geared 
toward  the  same  goals.  Reflecting  this  focus,  the  law  requires  the  alignment  of 
Eisenhower-assisted  professional  development  activities  with  challenging  state  and  local 
curriculum  standards  and  student  assessments. 

■ The  Eisenhower  legislation  requires  the  coordination  of  supported  activities  with 
education  reform  and  professional  development  efforts  funded  by  federal,  state,  and 
local  governments  and  other  public,  private,  and  nonprofit  organizations  and 
associations.  Such  integration  of  Eisenhower-assisted  activities  with  other  reform  efforts 
would  presumably  strengthen  the  quality  of  those  activities  by  gearing  them  to 
challenging  standards  and  by  allowing  several  funding  sources  to  be  combined  to  design 
higher  quality  activities.  The  law’s  requirements  for  district  planning  of  professional 
development  activities,  for  co-funding  those  activities  with  funds  from  other  programs, 
and  for  IHE/NPOs  working  with  schools,  school  districts,  or  consortia  of  districts,  all 
promote  linkages  between  Eisenhower- assisted  activities  and  those  funded  from  other 
sources. 
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■ The  law  encourages  the  use  of  continuous  improvement  strategies  to  plan  and 
improve  Eisenhower-assisted  professional  development  activities,  through  careful 
goal-setting  and  monitoring.  The  provisions  are  intended  to  foster  purposeful  planning 
and  tracking  of  progress  by  states  and  localities,  supported  by  state  and  district 
performance  indicators,  needs  assessment,  and  evaluation. 

■ The  law  also  encourages  teacher  participation  in  decision-making  about  how  to  use 
funds  from  the  Eisenhower  program.  The  law  encourages  districts  to  include  teachers 
in  developing  both  the  state  and  local  plans  for  professional  development,  which  are 
required  by  the  law,  and  the  legislation  also  requires  teachers’  participation  in  district 
needs  assessment. 

Some  of  the  key  goals  of  the  Eisenhower  program  are  summarized  in  a set  of  performance 
indicators  prepared  by  ED,  as  required  by  the  Government  Performance  and  Results  Act  (GPRA). 
This  evaluation  addresses  four  of  ED’s  performance  indicators  for  the  Eisenhower  program,  which 
concern  teachers’  skills  and  classroom  instruction,  sustained  professional  development,  high-quality 
professional  development,  and  participation  of  teachers  from  high  poverty  schools.16  In  reporting  the 
results  of  the  evaluation,  we  use  our  data  to  describe  how  well  the  program  is  performing  on  each  of 
the  indicators. 


The  National  Evaluation  of  the  Eisenhower  Program 

The  evaluation  is  based  on  three  strands  of  data  collection. 

■ The  National  Profile.  This  strand  provides  information  about  program  goals,  strategies, 
operations,  and  activities  nationwide.  During  the  1997-98  school  year,  AIR  conducted 
telephone  interviews  with  a national  probability  sample  of  Eisenhower  coordinators  in 
363  school  districts  and  SAHE-grantee  project  directors  in  92  institutions  of  higher 
education  or  nonprofit  organizations  (IHE/NPOs).  We  also  collected  data  from  a mail 
survey  of  a national  probability  sample  of  1,027  teachers  who  participated  in  657 
Eisenhower-assisted  activities.17  We  use  these  Teacher  Activity  Survey  data  to  describe 
the  types  of  professional  development  supported  with  Eisenhower  funds  and  to  compare 
activities  sponsored  by  school  districts  to  those  sponsored  by  SAHE  grantees. 

■ The  Case  Studies.  This  strand  provides  detailed  information  about  how  the  Eisenhower 
program  operates  in  selected  states,  school  districts,  and  schools.  During  the  1997-98 
school  year,  AIR  conducted  In-Depth  Case  Studies  in  10  school  districts — two  school 
districts  in  each  of  five  states:  Kentucky,  New  York,  Ohio,  Texas,  and  Washington. 
Those  case  studies  supplement  six  Exploratory  Case  districts  visited  during  the  first  year 
of  the  evaluation. 

■ The  Longitudinal  Study  of  Teacher  Change.  This  strand  examines  the  effects  of 
Eisenhower-assisted  and  other  professional  development  on  teaching  practice  in 
mathematics  and  science.  In  each  of  the  10  districts  visited  for  the  in-depth  case  studies, 
we  interviewed  and  conducted  classroom  observations  of  teachers  in  three  schools,  for  a 
total  of  30  schools.  We  also  surveyed  all  teachers  who  teach  mathematics  or  science  in 
those  schools  at  three  points  in  time,  gathering  detailed  information  about  instruction 
during  the  1996-97,  1997-98,  and  1998-99  school  years. 
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These  multiple  strands  of  data  are  designed  to  produce  an  integrated  portrait  of  the 
Eisenhower  program  and  are  based  on  a variety  of  research  methods  and  relying  on  data  from  groups 
of  individuals  who  view  Eisenhower-assisted  activities  from  different  vantage  points.  Data  from  our 
telephone  interviews  with  district  program  coordinators  and  SAHE-grantee  project  directors,  for 
example,  are  backed  up  by  teacher-participant  accounts  of  what  they  experienced.  In  addition,  the 
survey  results  are  cross-validated  through  case  study  data  that  are  rich  in  potential  to  explain  the 
quantitative  results  from  the  surveys.  Finally,  although  our  national  data  on  the  effects  of 
participation  in  Eisenhower-assisted  activities  on  change  in  teaching  practice  are  based  on  teacher 
self-reports  and  do  not  provide  direct  estimates  of  change  over  time,  data  from  the  second  and  third 
waves  of  the  longitudinal  study,  to  be  examined  in  the  evaluation’s  third  report,  will  provide 
additional  information  on  teacher  change.18 


Effectiveness  of  Eisenhower- assisted  Professional 
Development  Activities 


To  measure  the  effectiveness  of  Eisenhower-assisted  professional  development  activities,  we 
asked  teachers  to  report  on  the  extent  to  which  participation  enhanced  their  knowledge  and  skills  in 
six  domains:  (1)  in-depth  knowledge  of  mathematics/science;  (2)  curriculum;  (3)  instructional 
methods;  (4)  approaches  to  assessment;  (5)  use  of  technology;  and  (6)  approaches  to  diversity. 


♦ Overall,  the  results  on  effectiveness  are  mixed.  When 
asked  directly,  many  teachers  in  SAHE-grantee  activities 
report  that  participation  in  Eisenhower-assisted 
professional  development  led  to  enhanced  knowledge  and 
skills  and  changes  in  their  classroom  teaching  practice. 

Somewhat  fewer  teachers  in  districts  report  that 
participation  in  Eisenhower-assisted  activities  led  to  these 
positive  teacher  outcomes.  (See  Exhibit  ES.l.) 

=>  Forty-eight  percent  of  teachers  in  district  activities  and 
68  percent  of  teachers  in  SAHE-supported  activities 
report  enhanced  in-depth  knowledge  of  mathematics  or 
science;  and  63  percent  of  teachers  in  district  activities 
and  79  percent  of  teachers  in  SAHE-supported  activities 
report  enhanced  knowledge  of  instructional  methods. 

=>  Percentages  are  lower  for  the  use  of  technology  and 

approaches  to  diversity.  (About  24  percent  of  teachers  in  district  activities  and  50  percent  in 
SAHE-grantee  activities  report  enhanced  knowledge  and  skills  in  technology;  26  percent  in 
district  activities  and  35  percent  in  SAHE-grantee  activities  report  enhanced  knowledge  and 
skills  in  approaches  to  diversity.) 


Indicator.  Teachers'  Skills 
and  Classroom  Instruction. 
By  1998,  more  than  50  percent 
of  a sample  of  teachers  will 
show  evidence  that 
participation  in  Eisenhower- 
assisted  professional 
development  has  resulted  in  an 
improvement  in  their 
knowledge  and  skills,  and  by 
2000,  more  than  60  percent  will 
show  such  evidence.  By  1999, 
more  than  50  percent  of  a 
sample  of  teachers  in  selected 
sites  will  show  evidence  that 
participation  in  Eisenhower- 
assisted  professional 
development  has  resulted  in 
improved  classroom 


=>  SAHE-grantee  activities  meet  ED’s  standard  for  changes  in  teachers’  knowledge  and  skills 
(see  box)  in  four  of  the  six  domains  measured  by  the  evaluation,  and  district  activities  meet 
this  standard  for  two  domains. 
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EXHIBIT  ES.1 


Percent  of  Teachers  Reporting  Enhanced  Knowledge  and  Skills  Due  to  Participation 
in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=731  to  750,  SAHE  Grantee  n=233  to  240) 


Areas  of  knowledge  and  skills 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  48  percent  of  the  teachers  who  participated  in  district  activities  report  their  in-depth  content 
knowledge  and  skills  have  been  enhanced  substantially  as  a result  of  professional  development.  Each  bar  and  the  number  on  the  top  of  it  represent 
the  percent  of  teachers  for  each  category. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the  program. 
“SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of  the  program. 
Due  to  difference  in  missing  data  across  items,  the  district  n ranges  from  73 1 to  750;  the  SAHE  Grantee  n ranges  from  233  to  240. 


♦ SAHE-grantee  activities  meet  benchmarks  for  reported  teacher  outcomes  set  by  other 
exemplary  professional  development  programs. 

=>  A comparison  of  our  data  for  SAHE-grantee  activities  with  the  results  obtained  for  34 

exemplary  summer  institutes  in  mathematics  and  science,  supported  by  the  National  Science 
Foundation,  the  Department  of  Education,  and  other  agencies,  indicates  that  teachers 
participating  in  SAHE-grantee  activities  report  enhancement  of  knowledge  and  skills  in 
mathematics  and  science  content  roughly  comparable  to  the  results  for  the  34  exemplary 
activities.19  Teachers  participating  in  district  Eisenhower  activities  show  weaker  results  than 
do  teachers  in  the  34  exemplary  activities. 


The  Quality  of  Eisenhower- assisted  Activities 


We  measured  the  quality  of  Eisenhower-assisted  activities  based  on  six  features  of  best 
practice  that  were  identified  through  a review  of  the  available  research  on  professional 
development  and  the  opinions  of  expert  practitioners: 

♦ the  form  or  organization  of  the  activity — that  is,  whether  the  activity  is  organized  as  a 
reform  type,  such  as  a study  group,  teacher  network,  mentoring  relationship,  committee 
or  task  force,  internship,  individual  research  project,  or  teacher  research  center,  in 
contrast  to  a traditional  workshop  or  conference; 

♦ the  duration  of  the  activity,  including  the  total  number  of  contact  hours  that  participants 
are  expected  to  spend  in  the  activity,  as  well  as  the  span  of  time  over  which  the  activity 
takes  place; 

♦ the  degree  to  which  the  activity  emphasizes  the  collective  participation  of  groups  of 
teachers  from  the  same  school,  department,  or  grade  level,  as  opposed  to  the  participation 
of  individual  teachers  from  many  schools; 

♦ the  degree  to  which  the  activity  has  a content  focus — that  is,  the  degree  to  which  the 
activity  is  focused  on  improving  and  deepening  teachers’  content  knowledge  in 
mathematics  or  science; 

♦ the  extent  to  which  the  activity  offers  opportunities  for  active  learning — that  is, 
opportunities  for  teachers  to  become  actively  engaged  in  the  meaningful  analysis  of 
teaching  and  learning,  for  example,  by  reviewing  student  work  or  obtaining  feedback  on 
their  teaching;  and 

♦ the  degree  to  which  the  activity  promotes  coherence  in  teachers’  professional 
development,  by  encouraging  the  continued  professional  communication  among  teachers, 
and  by  incorporating  experiences  that  are  consistent  with  teachers’  goals  and  aligned  with 
state  standards  and  assessments. 


♦ Most  Eisenhower-assisted  activities  are  traditional  in  form,  such  as  workshops,  courses,  or 
conferences.  Relatively  few  Eisenhower-assisted  activities  are  reform  types  of  activities, 
such  as  study  groups,  networks,  or  mentoring  relationships. 

=>  About  79  percent  of  teachers  in  district  activities  are  in  traditional  types  of  activities — 
primarily  workshops  and  conferences.  About  76  percent  of  teachers  in  SAHE-grantee 
activities  are  also  in  traditional  activities — primarily  workshops  and  college  courses. 

=>  About  22  percent  of  teachers  in  district  activities  and  26  percent  in  SAHE-grantee  activities 
are  in  reform  activities,  including  teacher  networks,  study  groups,  mentoring,  committees 
and  task  forces,  internships,  and  resource  centers. 
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♦ On  average,  SAHE-grantee  activities  are  of  longer  duration  than  district  activities. 


=>  The  average  amount  of  time  teachers  in  district  activities 
report  spending  in  Eisenhower-assisted  activities  is  25 
hours,  compared  to  51  hours  for  teachers  in  SAHE  grantee 
activities.  The  average  length  of  district  activities,  in 
hours,  has  approximately  doubled  since  the  last  evaluation 
was  conducted  in  1988-89.21 

=>  A higher  percentage  of  SAHE-grantee  activities  than 
district  activities  span  an  extended  period  of  time.  For 
example,  46  percent  of  teachers  in  SAHE-grantee 
activities  are  in  activities  lasting  at  least  six  months,  including  2 percent  lasting  more  than 
one  year,  20  percent  lasting  10-12  months,  and  24  percent  lasting  6-9  months.  (See  Exhibit 
ES.2.)  Twenty  percent  of  teachers  in  district  activities  are  in  activities  lasting  at  least  six 
months. 

=>  ED's  performance  indicator  for  sustained  professional  development  requires  at  least  35 
percent  of  teachers  to  be  in  activities  that  extend  over  the  school  year.  If  "lasting  at  least  six 
months"  is  used  as  the  standard,  then  district  activities  do  not  yet  meet  the  standard,  while 
many  SAHE-grantee  activities  exceed  the  standard  by  a substantial  amount. 22 


Indicator.  Sustained 
Professional  Development. 

By  1998,  35  percent  of  teachers 
participating  in  district-level 
Eisenhower-assisted  activities 
will  participate  in  activities  that 
are  a component  of  professional 
development  that  extends  over 
the  school  year;  by  2000,  over 
50  percent  will. 


EXHIBIT  ES.2 


Time  Span  of  Eisenhower-assisted  Activities,  as  Reported  by  Teachers 
(District  n=766,  SAHE  Grantee  n=244) 


Over  - 
a year 

2% 

2% 

2% 

10-12  - 
months 

8% 

7£ 

15% 

6-9  - 
months 

10,% 

9% 

11% 

2-5  - 
months 

7& 

5% 

13% 

a month  - 

9% 

7£ 

1% 

a week  - 

8% 

8% 

9% 

2-4  _ 

27% 

31% 

16% 

days 
a day  - 

17% 

19% 

8% 

less  than  . 
a day 

12,% 

12% 

10% 

Overall  Traditional  Reform 


Districts 


2% 

1% 

* 7% 

20% 

16% 

33% 

24,% 

26% 

17% 

10% 

12% 

6% 

11% 

10% 

12% 

14% 

16% 

8% 

10% 

9% 

10% 

7% 

8% 

6% 

2% 

3% 

2% 

Overall 

Traditional 

Reform 

SAHE  Grantees 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit'.  The  first  column  shows  that  2 percent  of  the  teachers  who  participated  in  district  activities  were  engaged  in  the 
activity  over  more  than  a year.  Each  dot  represents  one  teacher.  If  more  than  one  teacher  reported  the  same  span,  the  teachers  are  displayed 
in  a horizontal  line  with  length  proportional  to  the  number  of  teachers.  Each  column  represents  the  distribution  for  a particular  group  of 
teachers.  The  number  on  the  top  of  each  line  is  the  percent  of  teachers  participating  in  the  corresponding  time  span. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower- assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower- assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


♦ Relatively  few  district  or  SAHE-grantee  activities  emphasize  the  collective  participation  of 
teachers  from  the  same  department,  grade  level,  or  school.  District  activities  give  more 
emphasis  to  collective  participation  than  do  SAHE-grantee  activities. 

=>  Twenty  percent  of  teachers  in  district  Eisenhower-assisted  activities  report  participating  with 
other  teachers  in  their  department  or  grade  level,  compared  to  7 percent  of  teachers  in  SAHE- 
grantee  activities;  and  19  percent  of  teachers  in  district  activities  report  participating  with  all 
teachers  in  their  school,  compared  to  11  percent  of  teachers  in  SAHE-grantee  activities. 


♦ On  average,  SAHE-grantee  activities  give  more  emphasis  than  do  district  activities  to 
the  last  three  dimensions  of  quality — content  focus,  active  learning,  and  coherence. 


=>  Two  thirds  of  teachers  participating  in  SAHE-supported  Eisenhower  activities  participate  in 
activities  that  place  a major  emphasis  on  content,  compared  to  fifty-one  percent  of  teachers 
in  district  Eisenhower-assisted  activities. 


=>  The  percentage  of  teachers  reporting  a major  emphasis  on  each  of  18  separate  indicators  of 
active  learning  is  higher  for  SAHE  grantees  than  for  districts,  but  relatively  few  teachers  in 
either  district  or  SAHE-grantee  activities  report  some  elements  of  active  learning.  For 
example,  only  5 percent  of  teachers  in  district  activities  and  16  percent  of  teachers  in  SAHE- 
grantee  activities  report  that  their  teaching  was  observed  by  the  activity  leader. 

=>  Teachers  in  both  district  and  SAHE-grantee  activities  report  that  the  activities  have  elements 
that  promote  coherence  with  other  aspects  of  their  professional  experiences.  For  example, 
more  than  three  quarters  of  teachers  in  both  district  and  SAHE-grantee  activities  report  that 
their  activities  are  aligned  with  state  and  district  standards.  More  teachers  in  SAHE-grantee 
than  district  activities,  however,  report  that  their  Eisenhower  activities  built  on  prior 
professional  development  (39  percent  compared  to  3 1 percent)  or  were  followed  up  with  later 
activities  (70  percent  compared  to  53  percent). 


ED's  performance  indicator  for  high-quality 
professional  development  requires  that  at  least 
50  percent  of  teacher  participants  be  in 
activities  reflecting  "best  practice."  The  data 
collected  by  the  evaluation  show  that  district 
and  SAHE-grantee  activities  meet  this  standard 
for  some  dimensions  of  high  quality  but  not 
others.  For  example,  as  discussed  above,  more 
than  50  percent  of  teachers  in  SAHE-grantee 
and  district  Eisenhower  activities  are  in  activities  that  place  a major  emphasis  on 
mathematics  and  science  content,  and  thus  ED’s  “best  practice”  standard  for  high  quality  is 
met.  But  districts  do  not  meet  the  high  quality  standard  for  any  of  the  characteristics  of 
active  learning,  and  SAHE  grantees  meet  the  standard  for  only  a few  characteristics  of  active 
learning. 


Indicator.  High  Quality.  By  1998,  more 
than  50  percent  of  teachers  participating  in 
district-level,  Eisenhower-assisted 
professional  development  activities  will 
participate  in  activities  reflecting  best 
practices,  including  a focus  on  continuous 
improvement.  By  2000,  more  than  75  percent 
will. 
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♦ There  is  enormous  variability  in  the  quality  of  Eisenhower-assisted  activities;  thus,  some 
districts  are  providing  the  same  kinds  of  high-quality  professional  development  activities 
that  SAHE  grantees  provide. 

=>  For  example,  although  many  district  activities  are  short,  2 percent  of  teachers  in  district 
activities  are  in  activities  spanning  more  than  one  year,  8 percent  are  in  activities  lasting  10- 
12  months,  and  10  percent  are  in  activities  lasting  6-9  months.  (See  Exhibit  ES.2.) 

=>  That  some  districts  provide  activities  of  extended  duration,  with  collective  participation,  a 
major  focus  on  content  knowledge,  a major  emphasis  on  active  learning,  and  coherence  with 
teachers'  other  experiences,  represents  an  “existence  proof’  that  it  is  possible  for  districts  to 
provide  such  activities. 

We  do  not  know  whether  all  districts  possess  the  organizational  capacity  and  staffing  to  provide 
high-quality  professional  development.  Districts  lacking  the  capacity  to  plan  and  implement 
high-quality  professional  development  themselves  may  well  be  able  to  purchase  it  (e.g.,  from 
institutions  of  higher  education),  if  they  are  aware  of  and  insist  upon  the  dimensions  of  quality 
we  have  identified. 


♦ The  average  differences  in  quality  observed  between  district  and  SAHE-supported 
activities  are  associated  with  corresponding  differences  in  cost. 

^ SAHE  grantees  spend  more  than  twice  as  much  per  teacher  participant  as  do  districts.  We 
estimate  that  SAHE  grantees  spend  about  $512  per  participation,  in  comparison  to  $185  per 
participation  for  districts.23 


It  is  unclear  why  SAHE  grantees  spend  more  money  per  teacher  participation  than  do  districts. 
The  competitive  process  of  the  SAHE  grants  may  reward  proposals  that  offer  professional 
development  with  the  six  quality  features,  which  are  expensive  to  provide.24  In  addition, 
compared  to  SAHE  grantees,  districts  may  feel  a responsibility  to  provide  professional 
development  to  all  of  their  teachers.  This  may  push  them  in  the  direction  of  professional 
development  with  lower  costs  per  participation. 


♦ Districts  have  met  ED’s  standard  for  participation  of  teachers  from  high  poverty  schools, 
but  there  remains  room  for  improvement  (see  box). 

SAHE  grantees  have  not  met  ED’s  standard. 

=>  Teacher  participations  in  district  Eisenhower-assisted 
activities  are  slightly  more  likely  to  be  from  high-poverty 
schools  than  are  teachers  in  the  national  teaching  force  as 
a whole  (23  percent  compared  to  21  percent). 

=>  Teacher  participations  in  SAHE-grantee  activities, 

however,  are  less  likely  to  be  from  high-poverty  schools 

than  are  teachers  in  the  national  teaching  force  (13  percent  compared  to  21  percent). 


Indicator . High-poverty 
Schools.  The  proportion  of 
teachers  participating  in 
Eisenhower-assisted  activities 
who  teach  in  high-poverty 
schools  will  exceed  the 
proportion  of  the  national 
teacher  pool  who  teach  in  high- 
poverty  schools. 
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To  some  extent,  the  results  for  districts  are  encouraging:  they  indicate  that  the  program  has  been 
somewhat  successful  at  targeting  resources  on  teachers  of  disadvantaged  children.  But  the 
percentage  of  teachers  from  high-poverty  schools  served  by  the  program  is  only  modestly  higher 
than  the  rate  for  the  nation,  indicating  that  more  progress  might  be  made  in  this  area. 


Features  of  Professional  Development  That  Promote 

Improvements  in  Teaching 


We  used  our  national  probability  sample  of  teachers  who  participated  in  Eisenhower-assisted 
professional  development  to  estimate  the  strength  of  the  relationships  among  features  of  the 
professional  development  and  self-reported  teacher  outcomes — enhanced  knowledge  and  skills  and 
changed  teaching  practice. 


♦ Activities  with  the  six  features  of  high  quality — reform  type,  duration,  collective 

participation,  content  focus,  active  learning,  and  coherence — promote  better  self-reported 
outcomes  for  teachers  than  other  activities. 

=>  The  results  provide  empirical  support  for  the  contention  that  professional  development  that 
emphasizes  content  knowledge,  active  learning,  and  coherence  leads  to  teachers  reporting 
enhanced  knowledge  and  skill  and  changes  in  teaching  practices.  (See  Exhibit  ES.3.) 

=>  The  results  also  show  that  activities  of  longer  duration  and  activities  that  encourage  collective 
participation  of  teachers  tend  to  place  more  emphasis  on  content  than  other  activities,  provide 
more  opportunities  for  active  learning,  and  provide  more  coherent  professional  development. 
These  features  in  turn  promote  positive  teacher  outcomes.  Thus,  our  results  suggest  that  the 
emphasis  in  the  legislation  on  sustained  and  intensive  professional  development  is 
appropriate. 

=>  As  suggested  in  the  literature  on  professional  development,  reform  types  of  activities  tend  to 
produce  more  positive  reported  outcomes  than  traditional  types,  but  our  results  indicate  that 
this  effect  is  largely  indirect.  That  is,  reform  activities  tend  to  produce  better  outcomes 
primarily  because  they  tend  to  be  of  longer  duration.  Traditional  and  reform  activities  of  the 
same  duration  tend  to  have  the  same  effects  on  reported  outcomes. 

=>  Our  analyses  show  that  the  difference  between  districts  and  SAHE  grantees  in  reported 
enhancement  of  knowledge  and  skills  and  reported  change  in  teaching  practice  is  explained 
almost  entirely  by  the  fact  that  SAHE  grantees  place  a greater  emphasis  upon  duration, 
subject-matter  content,  active  learning,  and  coherence.  In  short,  if  districts  placed  a greater 
emphasis  upon  these  characteristics,  we  hypothesize  that  teachers  would  report  their 
knowledge  and  skill  enhanced  to  the  same  extent  as  teachers  in  SAHE-supported  activities, 
and  that  they  would  be  just  as  likely  to  report  changing  their  teaching  practice. 
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District  Management  of  Eisenhower-assisted  Activities 


We  now  turn  from  describing  teachers’  experiences  in  Eisenhower-assisted  activities  to  a 
description  of  the  ways  that  districts  manage  and  operate  their  Eisenhower  programs.  Each  district 
receiving  Eisenhower  funds  generally  uses  the  funds  to  support  a collection  of  professional 
development  activities.  The  collection  of  professional  development  activities  that  a district  supports 
with  Eisenhower  funds  can  be  viewed  in  its  entirety  as  its  “portfolio’  of  Eisenhower-assisted 
professional  development  activities.  In  our  analysis,  we  examine  the  factors  that  influence  the 
quality  of  the  portfolios  of  professional  development  activities  that  districts  offer  and  the  extent  to 
which  districts  engage  in  efforts  to  target  teachers  of  high-need  students. 

The  measures  we  use  to  characterize  the  quality  of  a district’s  portfolio  of  Eisenhower- 
assisted  activities  are:  (1)  the  percent  of  the  districts’  participations  in  reform  types  of  activities,  (2) 
the  average  time  span  of  activities  (i.e.,  number  of  days,  weeks,  or  months),  (3)  the  number  of 
opportunities  for  active  learning  in  in-district  workshops  and  institutes,  and  (4)  the  degree  of 
collective  participation  in  in-district  workshops  and  institutes.25 

In  assessing  district  management  and  operations,  we  focus  on  the  role  of  several  provisions 
emphasized  in  the  legislation:  (1)  the  coordination  (co-funding)  of  Eisenhower-assisted  activities 
with  other  sources  of  funding  for  professional  development;  (2)  the  alignment  of  Eisenhower-assisted 
activities  with  state  and  district  standards  and  assessments;  (3)  the  participation  of  teachers  and 
school-level  staff  in  planning  Eisenhower-assisted  activities;  and  (4)  the  use  of  a process  of 
continuous  improvement,  including  monitoring  progress  against  measurable  objectives  and 
performance  indicators.  (See  page  4 for  more  detail  on  these  provisions.) 

♦ Districts’  use  of  co-funding,  alignment,  continuous  improvement,  and  teacher  involvement 
in  planning  leads  to  higher  quality  professional  development.  Thus,  the  requirements  of  the 
legislation  concerning  these  provisions  appear  to  be  appropriate. 

=>  Districts  that  engage  in  more  co-funding  of  Eisenhower  activities  with  other  programs  tend  to 
support  a greater  proportion  of  reform  types  of  activities  than  districts  that  engage  in  less  co- 
funding, and  they  tend  to  provide  more  opportunities  for  collective  participation. 

=>  In  addition,  districts  that  engage  in  more  co-funding  tend  to  engage  in  more  extensive 

continuous  improvement  efforts  and  they  tend  to  involve  teachers  more  widely  in  planning, 
both  of  which  are  related  to  increased  opportunities  for  active  learning. 

=>  Districts  that  align  professional  development  with  standards  and  assessments  are  more  likely 
to  offer  reform  types  of  activities. 

=>  In  addition,  the  districts  that  align  professional  development  with  standards  and  assessments 
are  more  likely  than  others  to  engage  in  continuous  improvement,  which  is  related  to 
increased  opportunities  for  active  learning. 
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These  results  lead  us  to  conclude  that  the  intentions  of  the  program  concerning  strategies  such  as 
co-funding,  alignment,  continuous  improvement,  and  teacher  involvement  in  planning  are 
appropriate,  in  the  sense  that  they  lead  to  higher  quality  professional  development. 


♦ Among  districts  where  federal  programs  operate  and  support  professional  development, 
districts  are  more  likely  to  co-fund  with  programs  that  have  a mathematics  and  science 
focus  than  with  other  federal  programs. 

=>  For  example,  as  Exhibit  ES.4  shows,  44  percent  of  teachers  are  in  districts  with  an  NSF- 
funded  State  Systemic  Initiative  (SSI);  of  those,  67  percent  are  in  districts  where  the  SSI 
supports  professional  development;  and  of  those,  66  percent  are  in  districts  where  the 
Eisenhower  project  co-funds  with  the  SSI.  Among  teachers  in  districts  in  which  Title  I, 
Part  A operates  and  funds  professional  development,  50  percent  are  in  districts  where  the 
Eisenhower  project  co-funds  with  Title  I;  proportions  are  smaller  for  other  Department  of 
Education  programs.26 


EXHIBIT  ES.4 


Percent  of  Teachers  in  Districts  in  Which  Other  Federal  Programs  Operate,  Support 
Professional  Development,  and  Co-fund  with  Eisenhower-assisted  Activities 

(n=363) 


Federal  Program 

Percent  of  teachers  in 
districts  where  federal 
program  operates  in 
state  or  district 

Percent  of  teachers  in 
districts  where  federal 
program  supports 
professional 
development  in  district 

In  districts  where 
federal  program 
operates  and  supports 
professional 
development,  percent 
of  teachers  in  districts 
that  co-fund  with  other 
programs 

National  Science  Foundation 

State  Systemic  Initiative 

44 

67 

66 

Urban  Systemic  Initiative 

17 

89 

86 

Rural  Systemic  Initiative 

4 

78 

28 

Local  Systemic  Change  Initiative 

12 

91 

76 

Department  of  Education 

Title  1,  A (Disadvantaged) 

91 

91 

50 

Title  1,  C (Migrant  Children) 

40 

63 

35 

Title  III  (Technology) 

29 

91 

39 

Title  VI  (Innovative  Strategies) 

77 

73 

48 

Title  VII  (Bilingual) 

47 

82 

28 

Title  IX  (Indian) 

19 

57 

5 

IDEA  (Disabilities) 

77 

85 

22 

Goals  2000 

86 

73 

34 

SchooI-to-Work 

77 

85 

28 

Perkins  (Vocational) 

74 

83 

19 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  Forty-four  percent  of  teachers  are  in  districts  where  coordinators  report  that  a Statewide  Systemic  Initiative  operated  in 

their  state  in  1997.  Of  those,  67  percent  funded  professional  development  during  1997,  and  of  those,  66  percent  co 

-funded  professional 

development  with  Eisenhower. 
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♦ Most  districts  report  that  their  Eisenhower-assisted  professional  development  activities  are 
aligned  with  standards  and  assessments. 

=>  Alignment  is  more  likely  to  be  reported  for  state  standards  and  assessments  than  it  is  for  district 
standards  and  assessments,  and  more  likely  to  be  reported  for  mathematics  than  for  science. 

=>  Alignment  is  more  likely  for  standards  than  for  assessments.  For  example,  according  to 

district  coordinators,  85  percent  of  teachers  are  in  districts  where  the  Eisenhower  professional 
development  activities  are  aligned  with  district  mathematics  standards.  In  contrast,  69 
percent  of  teachers  are  in  districts  where  the  Eisenhower  professional  development  activities 
are  aligned  with  district  mathematics  assessments. 

The  greater  alignment  for  standards  than  for  assessments  may  reflect  the  fact  that  the  Elementary 
and  Secondary  Education  Act  (ESEA)  requirement  of  aligned  assessments  had  not  yet  gone  into 
effect  at  the  time  we  collected  our  data. 

♦ Almost  all  teachers  are  in  districts  that  report  involving  teachers  in  planning  professional 
development. 

=>  Ninety-nine  percent  of  teachers  are  in  districts  that  involve  teachers  in  district-level  planning 
of  Eisenhower-assisted  activities,  and  62  percent  are  in  districts  in  which  teachers  are 
involved  in  a formal  school-level  committee  to  plan  professional  development. 

These  findings  lead  us  to  conclude  that  the  provision  of  the  law  that  supports  teacher 
participation  in  planning  is  relatively  well  implemented.  Still,  our  case  study  data  reveal  that  the 
Eisenhower  legislation's  80-20  rule,  which  calls  for  at  least  80  percent  of  funds  to  be  used  "in  a 
manner  that  is  determined  by. . . teachers  and  staff'  and  "to  the  extent  practicable,  takes  place  at 
the  individual  school  site,"  is  not  well  understood  and  is  sometimes  not  even  known  to  exist.  In 
particular,  it  is  not  clear  if  the  provision  means  that  planning  should  involve  teachers  and  occur  at 
the  school,  or  whether  it  is  designed  to  encourage  teacher  involvement  in  district-level  planning. 

♦ The  use  of  performance  indicators  to  guide  the  continuous  improvement  process  is  not  yet 
widespread  in  districts  that  receive  Eisenhower  funds. 

=>  Fewer  than  one  in  five  teachers  are  in  districts  that  collect  data  on  performance  indicators 
established  to  guide  district  professional  development  efforts. 

=>  Only  one  third  of  the  teachers  are  in  districts  that  have  developed  such  indicators. 

=>  The  majority  of  teachers  are  in  districts  whose  Eisenhower  coordinators  are  not  aware  that 
their  state  has  performance  indicators. 

Clearly,  the  development  of  state  and  district  performance  indicators  is  a goal  of  the  program  that 
is  not  generally  being  implemented,  at  least  not  yet.  There  are  other  ways  in  which  districts  show 
some  commitment  to  continuous  improvement.  Needs  assessments  through  teacher  surveys  and 
informal  conversations  are  common.  Nearly  every  district  reports  evaluating  its  professional 
development  activities.  These  evaluations,  however,  are  frequently  based  on  teacher  satisfaction 
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surveys  or  participation  counts.  It  is  less  common  to  use  observations  of  teachers’  subsequent 
classroom  practice  to  evaluate  the  effectiveness  of  Eisenhower  professional  development. 


♦ District  efforts  to  target  teachers  of  special  populations  of  students  have  met  with  very 
limited  success. 

=>  Districts  report  a strong  emphasis  on  recruiting  teachers  from  high-poverty,  low-achieving 
schools  and  slightly  less  emphasis  on  recruiting  teachers  from  Title  I schools. 

=>  Despite  these  reported  efforts  at  targeting,  and  even  though  districts  with  larger  numbers  of 
high-poverty  students  receive  more  funding,  teachers  from  high-poverty  schools  are  only 
slightly  more  likely  than  others  to  participate  in  Eisenhower-funded  activities. 

Although  districts  have  met  the  Department  of  Education’s  standard  for  targeting  (see  page  10), 
current  targeting  strategies  apparently  have  limited  effectiveness.  Perhaps  these  results  are 
explained,  in  part,  by  the  fact  that  most  participants  in  Eisenhower-assisted  activities  are 
volunteers.  It  is  possible  that  districts  do  not  have  adequate  strategies  to  shape  the  incentives  and 
constraints  that  determine  which  teachers  volunteer. 


♦ Generally,  larger  districts  are  more  likely  to  manage  their  portfolios  better  and  to  provide 
higher  quality  professional  development  than  are  smaller  districts. 

=>  Larger  districts  are  more  likely  to  align  their  professional  development  with  standards  and 
assessments,  to  co-fund  their  projects,  and  to  have  a greater  commitment  to  continuous 
improvement;  they  also  provide  activities  of  longer  duration,  with  more  opportunities  for 
collective  participation  and  active  learning.27 

Perhaps  large  districts  have  a better  infrastructure  and  more  capacity  than  small  districts,  which 
may  enable  them  to  provide  higher  quality  professional  development.  Large  districts  also  may 
have  a greater  variety  of  funding  sources  in  addition  to  Eisenhower,  increasing  opportunities  for 
co-funding,  and  perhaps  creating  a complexity  that  demands  a commitment  to  continuous 
improvement. 


SAHE-grantee  Management  of  Eisenhower- assisted 

Activities 


SAHE  grantees  are  subject  to  the  same  provisions  for  alignment,  continuous  improvement, 
and  targeting  that  districts  must  follow.  However,  SAHE  grantees  also  are  subject  to  some  additional 
requirements  or  guidelines.  In  particular,  the  1994  reauthorization  emphasizes  the  importance  of 
coordination  between  SAHE  grantees  and  districts  in  planning  and  providing  professional 
development.  Each  SAHE  grantee  is  required  to  enter  into  an  agreement  with  one  or  more  districts 
for  the  provision  of  professional  development.  Further,  SAHE-grantee  projects  are  shaped  by  the 
priorities  and  guidelines  that  the  SAHE  sets  in  structuring  its  competition  for  the  Eisenhower 
awards.28 
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♦ SAHE  grantees’  coordination  with  districts  and  continuous  improvement  efforts  lead  to 
higher  quality  professional  development. 

=>  SAHE  grantees  that  coordinate  more  extensively  with  districts  (e.g.,  use  feedback 

mechanisms,  support  and  extend  district  activities,  co-fund  and  involve  districts  in  planning, 
implementing  and  monitoring)  provide  professional  development  that  spans  longer  amounts 
of  time  compared  to  SAHE  grantees  that  engage  in  less  coordination  with  districts. 

=>  Coordination  with  districts  also  has  a positive  effect  on  the  use  of  strategies  for  continuous 
improvement,  which,  in  turn,  increases  the  active  learning  opportunities  provided  in  SAHE- 
grantee  activities. 

=>  Coordination  also  is  related  to  greater  targeting  through  its  effect  on  continuous  improvement. 
These  results  for  coordination  parallel  those  found  for  the  district  sample. 

♦ SAHE  grantees  report  low  levels  of  implementation  of  some  types  of  coordination  with 
districts  but  high  levels  of  most  continuous  improvement  strategies. 

=>  SAHE  grantees  work  closely  with  districts  in  several  ways,  such  as  communicating  with 
district  staff  and  using  district  needs  assessments,  but  report  low  levels  of  other  key 
components  of  coordination,  such  as  co-funding  and  working  with  the  Eisenhower 
coordinator. 

=>  SAHE  grantees  report  moderately  high  levels  of  engagement  in  most  continuous 

improvement  efforts,  such  as  using  state  indicators,  conducting  needs  assessments  and 
evaluations;  few  SAHE  grantees,  however,  use  district  indicators  in  designing  their 
Eisenhower  activities. 

These  findings  suggest  that,  although  the  average  quality  of  SAHE-grantee  activities  is  high  on 
some  dimensions  (e.g.,  duration  and  content  focus),  quality  could  be  further  improved  by 
strengthening  the  coordination  between  SAHE  grantees  and  districts  and  giving  more  emphasis 
to  district  indicators. 

♦ On  average,  SAHE  grantee  projects  in  education  departments  offer  higher-quality 
activities  on  several  dimensions  than  projects  in  mathematics/science  departments. 

=>  Education  departments  sponsor  professional  development  activities  that  last  more  than  twice 
the  number  of  hours  and  span  a longer  time  period  than  activities  sponsored  by 
mathematics/science  departments. 

=>  Education  departments  engage  in  more  types  of  coordination  with  the  districts  from  which 
they  draw  their  teachers,  and  they  place  more  emphasis  on  continuous  improvement  than  do 
mathematics/science  departments. 
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=>  Education  department  projects  are  especially  strong  when  housed  in  research/doctoral 
universities,  where  they  are  more  likely  to  put  a strong  focus  on  content  and  provide 
numerous  opportunities  for  active  learning. 

The  superiority  of  professional  development  provided  through  education  departments,  in  contrast 
with  mathematics/science  departments,  may  be  due  to  the  fact  that  education  departments  have 
greater  expertise  in  student  and  teacher  learning  of  subject  matter,  as  well  as  experience  in 
coordinating  with  practitioners.  Faculty  in  these  departments  are  the  main  contributors  to  the 
literature  on  effective  professional  development  in  mathematics  and  science.  Projects  in 
mathematics/science  departments,  on  the  other  hand,  have  a strong  content  focus.  Others  have 
noted  that,  in  the  training  of  teachers,  quality  might  be  enhanced  by  better  collaboration  between 
educators  and  scientists.  Apparently,  the  same  can  be  said  for  continuing  teacher  education  in  the 
form  of  Eisenhower  professional  development.  Each  type  of  department  brings  unique  strengths 
to  designing  and  delivering  professional  development  that  could  be  complementary  in 
meaningful  collaboration. 


Implications  for  Professional  Development 


These  findings  of  the  National  Evaluation  of  the  Eisenhower  Professional  Development 
Program  have  a number  of  implications  for  professional  development  generally  and  for  future 
Eisenhower  legislation  and  program  operations  specifically. 

First,  the  program  should  seek  ways  to  encourage  the  use  of  all  features  of  professional 
development  that  are  related  to  teacher  knowledge  and  skills,  and  change  in  practice.  The 

program  should  continue  emphasizing  "sustained  and  intensive"  professional  development  activities. 
More  specifically,  this  evaluation  suggests  that  professional  development  should  focus  on  deepening 
teachers'  content  knowledge  and  knowledge  of  how  students  learn  particular  content,  on  providing 
opportunities  for  active  learning , and  on  encouraging  coherence  in  teachers'  professional 
development  experiences.  Eisenhower  professional  development  should  pursue  these  goals  using 
activities  of  greater  duration  and  collective  participation.  Clearly,  there  is  considerable  room  to 
increase  the  frequency  of  these  attributes  of  effective  professional  development  in  both  district  and 
SAHE-grantee  activities.  While  reform  professional  development  is  more  effective  than  traditional 
professional  development,  the  advantage  to  reform  activities  is  explained  primarily  by  greater 
duration. 

Second,  the  program  should  develop  additional  approaches  to  targeting  teachers  in 
high-poverty  schools.  While  increasing  the  participation  of  teachers  in  high-poverty  schools  is  an 
important  goal  of  the  legislation,  current  approaches  to  targeting  these  teachers  are  insufficient  to 
achieve  this  goal. 

Third,  federal  indicators  might  be  improved  by  specifying  more  clearly  the  dimensions 
of  high-quality  professional  development,  focusing  on  annual  improvements  rather  than 
absolute  levels,  and  setting  higher  standards.  The  federal  indicators  are  not  as  directive  as  they 
might  be  in  specifying  the  dimensions  of  high-quality  professional  development.  Our  analyses 
suggest  that  the  federal  indicators  should  guide  local  practice  toward  greater  emphasis  upon  content 
knowledge,  active  learning,  and  coherence,  and  that  districts  should  pursue  these  attributes  of 
professional  development  through  offering  activities  of  greater  duration  and  with  more  emphasis 
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upon  collective  participation.  In  addition,  several  of  the  federal  indicators  are  stated  in  terms  of 
absolute  levels  to  be  achieved  by  a specified  date  (e.g.,  “at  least  50  percent  of  teachers”).  It  would  be 
better  to  have  indicators  stated  in  terms  of  the  amount  of  improvement  needed  annually,  until  an 
acceptably  high  level  of  performance  is  reached.  Finally,  the  federal  indicators  set  some  standards 
that  are  too  low.  For  example,  in  targeting  teachers  from  high-poverty  schools,  the  standard  is 
simply  to  exceed  the  national  average.  Furthermore,  in  the  targeting  indicator,  participation  in 
Eisenhower  professional  development  is  stated  as  a dichotomous  variable  (participate  or  not),  while 
clearly  teachers  participate  in  Eisenhower-supported  professional  development  in  varying  amounts 
and  types. 

Fourth,  the  program  should  pay  attention  to  building  district  capacity  to  foster 
continuous  improvement  efforts.  If  using  data  to  make  decisions  is  to  be  a serious  endeavor,  then 
districts  may  need  assistance  in  determining  the  types  of  data  that  would  be  useful  and  in  interpreting 
them.  We  believe  local  evaluation  can  be  done  in  ways  much  stronger  than  current  practice,  which 
relies  heavily  upon  participation  counts  and  teacher  satisfaction  surveys.  We  recommend  that  local 
evaluation  of  professional  development  focus  on  instruction,  by  assessing  the  degree  to  which  the 
professional  development  is  characterized  by  well-implemented  attributes  of  high-quality 
professional  development:  a focus  on  content,  active  learning,  and  coherence,  delivered  with 
sufficient  duration  and  collective  participation.  Such  evaluations  might  be  based  on  a variety  of 
sources  of  data,  including  well-designed  surveys  of  participating  teachers,  and  the  observation  of 
teachers  to  assess  the  extent  to  which  they  have  made  appropriate  changes  in  their  instruction. 
Although  the  assessment  of  student  achievement  might  be  used  as  part  of  a balanced  evaluation  of 
professional  development,  we  suggest  that  local  evaluation  efforts  should  not  focus  on  assessing  the 
effects  of  professional  development  on  student  achievement.  There  are  simply  too  many  intervening 
variables  between  professional  development  experiences  and  subsequent  gains  in  student 
achievement  to  make  such  studies  feasible  at  the  local  level.  Further,  collecting  and  analyzing  high- 
quality  data  on  gains  in  student  achievement  is  expensive  and  requires  technical  skills  that  may  not 
be  present  at  the  local  level.  Clearly,  there  needs  to  be  more  research  that  looks  at  the  relationships 
between  features  of  professional  development  and  gains  in  student  achievement,  but  this  is  better 
done  as  a part  of  well-designed  major  research  studies. 

Fifth,  more  information  is  needed  on  the  characteristics  and  conditions  that  give  some 
districts  the  capacity  to  provide  high-quality  professional  development.  We  have  speculated  that 
districts  could  provide  the  same  types  of  professional  development  as  do  SAHE  grantees.  There  are 
existence  proofs  of  districts  doing  exactly  that.  What  we  do  not  know  from  our  analyses  is  the 
percentage  of  districts  that  have  the  capacity  to  provide  such  professional  development.  Our  analyses 
indicate  that  larger  districts  have  greater  capacity  and,  to  some  extent,  so  do  high-poverty  districts. 
This  larger  capacity  may  be  explained,  in  part,  by  their  larger  district  staff  and,  in  part,  by  their 
greater  Eisenhower  funds.  Consortia  that  tie  together  several  small  districts  into  one  unit  for 
providing  Eisenhower  professional  development  also  seem,  on  average,  more  effective  than  small 
districts.  The  issue  of  district  capacity  is  one  that  should  be  closely  monitored  in  future  research. 

Sixth,  districts  could  increase  the  quality  of  the  professional  development  they  provide 
by  focusing  their  Eisenhower  money  on  a small  number  of  teachers,  rather  than  spreading  it 
across  a large  number  of  teachers.  Not  surprisingly,  high-quality  professional  development  costs 
more  per  participant  than  does  lower  quality  professional  development.  Districts  may  feel  a greater 
responsibility  to  reach  a large  number  of  teachers  than  do  SAHE  grantees,  and  this  is  reflected  in  the 
cost  per  participant.  The  question  is,  should  districts  continue  to  spread  the  money  from  the 
Eisenhower  program  across  as  many  teachers  as  possible?  Or,  should  they  focus  the  money  on  a 
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small  number  of  teachers,  so  that  they  can  provide  higher  quality,  more  influential  professional 
development?  Our  results  suggest  the  money  should  be  focused.  This  recommendation  also  interacts 
with  the  finding  about  targeting.  More  effective  targeting  might,  at  the  same  time,  provide  a 
rationale  for  more  focused  expenditures. 

Seventh,  one  reason  that  SAHE-grantee  professional  development  is,  on  average,  of 
higher  quality  than  district  professional  development  may  be  that  SAHE  grantees  have  to 
compete  for  funds.  We  do  not  know  as  much  about  these  SAHE-sponsored  competitions  as  we 
would  like.  We  find  that  the  majority  of  SAHE  grantees  have  been  receiving  Eisenhower  support  for 
several  years.  Still,  in  all  cases,  IHE/NPOs  interested  in  Eisenhower  support  must  develop  a 
proposal  and  have  it  judged  worthy  of  funding.  In  contrast,  districts  receive  money  from  the  state 
educational  agency  on  a formula  basis,  with  no  competition.  Perhaps  having  districts  compete  for 
funds  would  push  them  toward  higher  quality  professional  development. 

Eighth,  there  is  considerable  evidence  that,  on  average,  education  departments  in 
institutions  of  higher  education  provide  higher  quality  professional  development  than  do 
mathematics  and  science  departments.  The  one  exception  is  that  mathematics/science  departments 
in  non-research/doctoral  institutions  focus  more  on  content  knowledge  than  do  education 
departments.  We  do  not  have  empirical  data  on  interdepartmental  collaboration,  but,  in  our  case 
studies,  we  did  not  see  much  evidence  of  these  two  types  of  departments  collaborating  and 
combining  their  expertise  to  provide  high-quality  professional  development.  Lack  of  collaboration 
between  education  and  mathematics/science  departments  is  a well -recognized  problem  in  preservice 
teacher  education  and  may  be  an  issue  for  in-service  education,  as  well. 

Ninth,  SAHE-grantee  projects  should  be  engaging  in  higher  levels  of  coordination  with 
districts,  as  called  for  in  the  legislation.  The  evaluation  shows  that  such  coordination  is  related  to 
the  provision  of  high-quality  professional  development.  Therefore,  more  attention  might  be  paid  to 
supporting  and  developing  opportunities  for  SAHE  grantees  to  coordinate  and  work  with  districts  in 
mutually  beneficial  ways — ways  that  allow  grantees  to  exercise  their  expertise  in  developing 
professional  development  projects  while  benefiting  from  district  expertise  in  serving  the  needs  of 
their  teachers  and  students. 

Finally,  the  evaluation  supports  the  importance  of  programs  that  fund  professional 
development  activities  within  specified  subject  areas.  During  the  past  15  years,  the  Eisenhower 
Professional  Development  Program  has  provided  continuous  support  for  professional  development 
activities  for  mathematics  and  science.  This  evaluation  highlights  the  importance  of  the  content 
focus  of  professional  development  activities  and  the  role  that  the  Eisenhower  program  has  played  in 
building  capacity  in  these  two  subjects  in  school  districts.  Generic  professional  development  that 
focuses  on  teaching  techniques  without  a content  focus  does  not  appear  to  be  effective.  If  the 
Congress  is  considering  expanding  the  program,  it  should  consider  creating  analogous  programs  in 
other  academic  subject  areas,  rather  than  eliminating  the  content  focus  on  mathematics  and  science. 
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considering  the  multistage  process  necessary  to  complete  the  sample.  District  coordinators  and  project  directors  in 
SAHE-grantee  institutions  had  to  submit  the  complete  list  of  professional  development  activities  provided  during  the 
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participants  in  outstanding  teacher  development  programs  reported  that  the  programs  enhanced  their  knowledge  and 
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S.,  & Hewson,  P.  (1996,  May).  Principles  of  effective  professional  development  for  mathematics  and  science 
education:  A synthesis  of  standards,  NISE  Brief  (V ol.  1).  Madison,  WI:  National  Institutes  for  Science  Education). 
The  research  literature  contains  a mix  of  large-  and  small-scale  studies,  including  intensive  case  studies  of  classroom 
teaching,  evaluations  of  programs  designed  to  improve  teaching  and  learning,  and  surveys  of  teachers  about  their 
preservice  and  in-service  preparation  and  in-service  professional  development  experiences.  In  addition,  there  is  a 
large  body  of  literature  describing  "best  practices"  in  professional  development,  drawing  on  expert  experiences. 
Despite  the  size  of  the  literature,  however,  relatively  little  systematic  research  has  been  conducted  on  the  effects  of 
professional  development  on  improvements  in  teaching  or  on  student  outcomes,  and  very  little  research  has  been 
conducted  on  the  relative  effects  of  alternative  forms  of  professional  development.  The  research  that  has  been 
conducted,  however,  along  with  the  experience  of  expert  practitioners,  does  provide  some  preliminary  guidance 
about  the  characteristics  of  high-quality  professional  development  (See,  in  particular,  Loucks-Horsley,  S.,  Hewson, 
P.  W.,  Love,  N.,  & Stiles,  K.  E.  (1998).  Designing  professional  development  for  teachers  of  science  and 
mathematics.  Thousand  Oaks,  CA:  Corwin  Press).  In  particular,  several  recent  studies  suggest  that  professional 
development  that  focuses  on  specific  mathematics  and  science  content  and  the  ways  students  learn  such  content  is 
especially  helpful  (Cohen,  D.  K.,  & Hill,  H.  C.  (1998).  Instructional  policy  and  classroom  performance:  The 
mathematics  reform  in  California  (RR-39).  Philadelphia:  Consortium  for  Policy  Research  in  Education;  Kennedy, 

M.  M.  (1998).  Form  and  substance  in  in-service  teacher  education  (Research  monograph  no.  13).  Arlington,  VA: 
National  Science  Foundation).  We  integrated  and  operationalized  the  ideas  in  the  literature  on  "best  practices"  in 
professional  development  to  create  a set  of  measures  or  scales  describing  the  six  features  of  Eisenhower-assisted 
activities  described  in  the  text. 

21  The  1988-99  evaluation  collected  data  on  duration  from  districts  rather  than  teachers,  so  a comparison  of  results 
from  the  1988-89  and  the  current  evaluation  should  be  interpreted  as  providing  an  indication  of  the  general 
magnitude  of  the  change  rather  than  a precise  numerical  estimate.  See  Knapp,  M.  S.,  Zucker,  A.  A.,  Adelman,  N. 

E.,  & St.  John,  M.  (1991,  February).  The  Eisenhower  Mathematics  and  Science  Education  Program:  An  enabling 
resource  for  reform  (summary  report ).  Menlo  Park,  CA:  SRI  International,  p.  109. 

22  The  indicator  requires  that  activities  “are  a component  of  professional  development  that  extends  over  the  school 
year.”  It  is  possible  that  some  short-term  Eisenhower  activities  are  linked  to  other  activities,  and  these  “sequences” 
of  activities  extend  over  the  school  year.  If  so,  the  percent  of  Eisenhower-assisted  activities  extending  more  than  six 
months  may  understate  the  percent  of  activities  that  “are  a component  of  professional  development  that  extends  over 
the  school  year.”  On  our  teacher  survey,  we  asked  whether  the  Eisenhower-assisted  activities  were  followed  up  with 
additional  activities  that  built  upon  earlier  work;  59  percent  of  teachers  in  district  activities  and  70  percent  of 
teachers  in  SAHE-grantee  activities  reported  that  the  Eisenhower-assisted  activities  in  which  they  participated  were 
followed  up  with  additional  activities.  We  have  no  information  on  the  duration  of  the  follow-up  activities,  but 
assumed  that  some  of  the  follow-up  activities  might  extend  over  the  school  year. 

23  A “participation”  is  a teacher  participant  in  an  Eisenhower-assisted  activity.  Teachers  who  participate  in  more 
than  one  activity  are  counted  separately  for  each  activity  in  which  they  participate.  The  dollar  per  participation 
figure  for  districts  includes  federal  Eisenhower  dollars  only  and  does  not  include  the  33  percent  matching 
requirement. 

24  We  were  not  able  to  conduct  a systematic  analysis  of  SAHE  competitions. 

25  The  data  from  our  national  sample  of  teachers  show  that  each  of  these  dimensions  is  related,  either  indirectly  or 
directly,  to  improvements  in  teachers’  knowledge  and  skills  and  changes  in  teaching  practice;  thus,  we  consider  each 
of  these  dimensions  as  an  indicator  of  high-quality  professional  development,  whether  it  has  a direct  effect  on 
teacher  outcomes,  or  operates  indirectly  (e.g.,  a reform  approach  affects  teacher  outcomes  indirectly  through  its 
effect  on  duration). 

26  We  have  no  information  on  the  scope  of  co- funding  (e.g.,  the  amount  of  money  contributed  by  a particular 
program  in  a cost-sharing  arrangement). 

27  Throughout  our  analyses  of  district  data,  we  tested  to  see  where  patterns  of  Eisenhower  support  for  professional 
development  differ  significantly  according  to  the  district  poverty  level  or  the  size  of  the  district.  All  of  our  analyses 
simultaneously  control  for  size  and  poverty,  so  any  effects  are  independent  of  one  another.  We  also  tested  for  the 
interaction  between  these  two  variables. 
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28  Relative  to  SEAs,  SAHEs  have  a smaller  number  of  grantees,  and  thus  may  be  able  to  monitor  their  grantees’ 
projects  to  help  ensure  the  implementation  of  quality  activities.  However,  we  did  not  examine  the  SAHE’s 
monitoring  role. 
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CHAPTER  1 


OVERVIEW  OF  THE  EISENHOWER  PROFESSIONAL 
DEVELOPMENT  PROGRAM  AND  THE  DESIGN  OF  THE 

NATIONAL  EVALUATION 


The  professional  development  of  teachers  is  a crucial  element  of  the  nation’s  efforts  to 
improve  education.  In  recent  years,  these  efforts  have  sought  to  foster  high  standards  for  teaching 
and  learning  for  all  of  the  nation’s  children,  and  almost  all  states  have  met  federal  requirements  for 
developing  challenging  statewide  content  standards.1  Such  standards  seek  a fundamental  shift  in 
what  students  learn.  However,  children’s  learning  will  be  transformed  only  if  high  standards  are 
reflected  in  teachers’  classroom  practice.  Education  reforms  will  not  succeed  without  teachers  who 
are  immersed  in  the  subjects  that  they  teach  and  who  know  how  to  foster  both  basic  knowledge  and 
advanced  thinking  and  problem  solving  among  their  students  (Loucks-Horsley  et  al.,  1998;  National 
Commission  on  Teaching  & America’s  Future,  1996). 

The  Eisenhower  Professional  Development  Program,  Title  II  of  the  Elementary  and 
Secondary  Education  Act  (ESEA),  is  the  federal  government’s  largest  investment  that  is  solely 
focused  on  developing  the  knowledge  and  skills  of  classroom  teachers.  The  program  is  key  to 
meeting  the  U.S.  Department  of  Education’s  objective  of  ensuring  that  a “talented  and  dedicated 
teacher  is  in  every  classroom  in  America”  (U.S.  Department  of  Education,  1999c).  Part  B of  the 
Eisenhower  Professional  Development  Program,  with  a 1999  appropriation  of  about  $335  million, 
provides  funds  through  state  education  agencies  (SEAs)  to  school  districts,  and  through  state 
agencies  for  higher  education  (S  AHEs)  to  institutions  of  higher  education  and  nonprofit 
organizations  (SAHE  grantees).  These  funds  primarily  support  professional  development  in 
mathematics  and  science.  Congress  and  the  U.S.  Department  of  Education  (ED)  currently  are 
considering  changes  to  the  Eisenhower  program  as  part  of  the  reauthorization  of  the  ESEA.  This 
report,  based  on  a national  evaluation  of  the  Eisenhower  program,  aims  to  provide  information  about 
how  the  program  is  operating  and  how  it  might  be  strengthened  to  inform  policymakers  as  they 
consider  changes  to  the  program  during  its  reauthorization. 

In  addition  to  informing  policymakers  about  the  Eisenhower  program,  a second  objective  of 
this  report  is  to  contribute  more  generally  to  knowledge  about  professional  development.  The 
Eisenhower  program  is  a source  of  funding  for  professional  development  activities,  not  a specific 
approach  to  professional  development.  Allowable  activities  are  wide-ranging  and  include  workshops 
and  conferences,  study  groups,  professional  networks  and  collaboratives,  task  force  work,  and  peer 
coaching.  Furthermore,  Eisenhower  funding  does  not  exist  in  a vacuum.  Eisenhower-assisted 


1 Title  I of  the  Elementary  and  Secondary  Education  Act  (ESEA)  required  that  by  the  1997-98  school  year,  each  state 
was  to  have  adopted  challenging  content  standards  in  at  least  reading  and  mathematics,  and  challenging  performance 
standards  that  describe  students’  mastery  of  the  content  standards  (U.S.  Department  of  Education,  1999b).  Forty- 
eight  states,  plus  Puerto  Rico  and  the  District  of  Columbia,  have  met  federal  requirements  for  developing  challenging 
statewide  content  standards.  Twenty-one  states,  plus  Puerto  Rico,  have  met  the  requirement  for  developing  student 
performance  standards.  States  are  not  required  to  submit  content  and  performance  standards  to  the  U.S.  Department 
of  Education — only  evidence  of  their  quality  and  rigor. 
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activities  also  may  receive  funding  through  states,  school  districts,  and  other  federal  programs. 
Therefore  the  information  in  this  report  about  the  quality  and  effects  of  Eisenhower-assisted 
activities  is  also  applicable  to  professional  development  funded  through  other  sources. 

Finally,  by  analyzing  features  of  professional  development  activities  and  their  effects  on 
teachers,  the  third  objective  of  this  report  is  to  gauge  the  extent  to  which  federal  support  for 
professional  development  of  teachers  enhances  the  quality  of  teaching  practice  in  the  nation’s 
schools.  Ultimately,  the  success  of  the  nation’s  efforts  to  improve  student  learning  depends  on 
efforts  to  improve  the  quality  of  teaching. 


The  Policy  Context  for  the  Reauthorized 
Eisenhower  Program 


The  Eisenhower  legislation  is  part  of  the  Elementary  and  Secondary  Education  Act  (ESEA), 
and  Congress  in  1994  took  a dramatically  different  direction  for  the  ESEA’s  programs.  In  particular, 
the  focus  of  Title  I,  the  cornerstone  of  the  ESEA,  was  to  change  from  a program  funding 
“remediation”  activities  to  one  that  aims  at  achieving  high  standards  for  all  students.2  Furthermore, 
the  “new”  Title  I was  intended  to  encourage  systemic  educational  reform  through  development  of 
state  standards  and  assessments,  as  well  as  by  supporting  school-based  education  reform  through 
easing  provisions  for  school  wide  projects.  As  part  of  ESEA,  the  Eisenhower  program  also  was  re- 
shaped in  1994  to  help  move  the  nation’s  children  toward  high  standards  by  improving  the  quality  of 
teaching. 

These  dramatic  changes  in  federal  education  programs  responded,  in  part,  to  public  concerns 
about  the  poor  performance  of  U.S.  students  compared  to  those  in  other  industrialized  countries.  In 
mathematics  and  science,  the  Third  International  Mathematics  and  Science  Study  (TIMSS)  suggests 
some  explanations  for  the  performance  of  U.S.  students.3  The  TIMSS  studies  found  that,  while  U.S. 
children  generally  have  grasped  “basics”  like  arithmetic,  many  are  not  learning  advanced 
mathematics  and  science  (Schmidt,  McKnight,  & Raizen,  1996;  Schmidt  & Valverde,  1997;  U.S. 
Department  of  Education,  1998b).  The  studies  also  found  that  curricula  in  U.S.  “schools  lacks 
coherence,  focus  and  rigor,  compared  to  that  of  other  countries”  (U.S.  Department  of  Education, 
1998b,  p.10).  Because  the  curriculum  frameworks  of  many  states  cover  many  topics — more  than  are 
taught  in  most  other  TIMSS  countries — teachers  often  cover  topics  superficially,  without  spending 
enough  time  for  students  to  master  the  topics.  The  curriculum  within  the  U.S.  also  is  less  demanding 


2 Title  I ot  the  Elementary  and  Secondary  Education  Act,  as  amended  by  the  Improving  America’s  Schools  Act,  is 
the  federal  government’s  largest  investment  in  K-12  education.  In  FY  1997,  Part  A of  the  program,  the  local 
education  agency  grants  program,  was  appropriated  at  $6.27  billion.  Most  of  these  funds  are  distributed  by  formula, 
based  on  the  number  of  children  who  live  in  poverty,  first  to  states  and  then  to  districts.  Established  in  1965  as  one 
of  the  cornerstones  of  President  Johnson’s  War  on  Poverty,  Title  I funds  educational  services  for  children  attending 
high-poverty  schools.  With  its  1994  reauthorization  of  the  program,  Congress  made  clear  its  intention  that  services 
provided  under  Title  I be  linked  to  high  state  and  local  standards. 

In  mathematics  and  science,  the  subjects  that  are  the  primary  focus  of  the  Eisenhower  program,  students  fall  behind 
during  the  course  of  their  school  years.  According  to  TIMSS,  U.S.  fourth-grade  students  are  among  the  best  in  the 
world  in  science,  and  above  the  international  average  in  mathematics.  By  high  school,  U.S.  students  score  near  the 
bottom  of  TIMSS  nations  in  both  subjects. 


than  that  of  many  other  TIMSS  countries.  For  example,  middle  school  students  in  the  U.S.  are 
reviewing  arithmetic  and  introductory  science  while  their  peers  in  other  countries  are  studying 
algebra  and  geometry,  physics,  and  chemistry.4  While  curricula  that  emphasize  basic  skills  without 
also  emphasizing  more  advanced  skills  may  have  been  sufficient  in  a manufacturing  economy,  such 
curricula  appear  not  to  be  sufficient  in  today’s  or  tomorrow’s  work  place. 

Addressing  deficiencies  in  student  performance  and  in  the  curricula  that  students  cover 
requires  improvements  at  all  levels  of  the  U.S.  education  system  (Fuhrman,  1993;  Goertz,  Floden,  & 
O’Day,  1996;  Kahle,  1997;  Lee,  1997;  Loucks-Horsley,  1997;  Smith  & O’Day,  1991;  Webb,  1997a, 
1997b).  Such  improvements  are  even  more  necessary  because  systemic  reform  efforts  are  attempting 
to  raise  standards  of  performance  for  all  children,  especially  those  from  diverse  backgrounds  who 
have  traditionally  performed  poorly  in  school.  Preparation  of  all  children  for  an  increasingly 
knowledge-based  economy  requires  a dramatic  overhaul  of  all  aspects  of  the  education  system, 
according  to  proponents  of  systemic  reform. 

In  response  to  public  concerns  about  education,  state  and  local  governments  have  taken  steps 
to  increase  children’s  achievement  in  school.  Many  states  and  school  districts  have  adopted  rigorous 
content  standards,  as  well  as  student  performance  standards,  which  describe  the  breadth  and  depth  at 
which  students  should  master  content  (American  Federation  of  Teachers,  n.d.;  Blank  & Pechman, 
1995;  National  Education  Goals  Panel,  1995;  Porter,  Archbald,  & Tyree,  1991;  Porter,  Smithson,  & 
Osthoff,  1994).  The  federal  government,  too,  has  moved  to  support  states  in  their  development  of 
content  and  student  performance  standards.  In  addition  to  the  Improving  America’s  Schools  Act  of 
1994  which  reauthorized  the  ESEA,  the  Goals  2000:  Educate  America  Act,  enacted  in  1994, 
provides  grants  to  states  to  support  systemic  reform  initiatives.  The  National  Science  Foundation 
(NSF)  also  has  invested  heavily  in  supporting  systemic  education  reform  initiatives  in  mathematics 
and  science  in  states,  urban  and  rural  areas,  and  school  districts.5 

National,  state,  and  local  efforts  to  improve  education  are  intended  to  create  a fundamental 
shift  in  what  students  learn  and  how  they  are  taught.  The  success  of  such  ambitious  education 
reform  initiatives  hinges,  in  large  part,  on  the  qualifications  and  effectiveness  of  teachers.  Thus,  if 
children  are  to  achieve  at  levels  demanded  by  the  high  standards  adopted  by  states  and  districts, 
teachers  will  have  to  help  them  do  so.  Teachers  are  necessarily  at  the  center  of  reform,  for  they  must 
carry  out  the  demands  of  reform  in  the  classroom  (Cuban,  1990).  As  a result,  teacher  professional 
development  is  a major  focus  of  systemic  reform  initiatives  (Corcoran,  1995;  Corcoran,  Shields,  & 
Zucker,  1998). 


4 TIMSS  is  the  largest  international  comparison  of  education  ever  done.  In  1995,  TIMSS  tested  the  mathematics  and 
science  knowledge  of  half  a million  students  in  three  different  grade  levels  in  4 1 countries. 

5 The  National  Science  Foundation  sponsors  four  systemic  initiatives:  Statewide  Systemic  Initiatives,  concerned 
with  state-level  change;  Urban  Systemic  Initiatives  (replaced  in  1999  by  the  Urban  Systemic  Program),  for  identified 
urban  areas  meeting  minimum  size  requirements;  Rural  Systemic  Initiatives,  intended  to  ensure  rural  areas  access  to 
the  technology  and  other  educational  reform  efforts  of  more  populous  areas;  and  Local  Systemic  Change  (formerly 
Local  Systemic  Initiatives),  primarily  concerned  with  teacher  in-service  training  and  development.  NSF’s  systemic 
initiatives  are  generally  funded  in  five-year  increments;  SSI  awards  can  be  up  to  $2  million  per  year  and  USI  grants 
are  between  $400,000  and  $3  million  per  year. 
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To  carry  out  the  demands  of  education  reform,  teachers  must  be  immersed  in  the  subjects 
they  teach,  and  have  the  ability  both  to  communicate  basic  knowledge  and  to  develop  advanced 
thinking  and  problem-solving  skills  among  their  students  (Loucks-Horsley  et  al.,  1998;  National 
Commission  on  Teaching  and  America’s  Future,  1996).  The  central  elements  of  systemic  reform — 
high  standards,  curriculum  frameworks,  and  new  approaches  to  assessment  aligned  to  those 
standards— generate  new  expectations  for  teachers’  classroom  behaviors,  as  well  as  for  student 
performance  (Bybee,  1993;  National  Council  for  Teachers  of  Mathematics,  1991;  National  Research 
Council,  1996;  Webb  & Romberg,  1994).  To  help  students  learn  complex  knowledge  and  skills, 
teachers  must  understand  the  disciplines  that  they  teach,  as  well  as  how  children  learn  these 
disciplines. 

However,  while  teachers  generally  support  high  standards  for  teaching  and  learning,  many 
teachers  are  not  prepared  to  implement  teaching  practices  based  on  high  standards  (Cohen,  1990; 
Elmore  & Consortium  for  Policy  Research  in  Education,  1996;  Elmore,  Peterson,  & McCarthy, 

1996;  Grant,  Peterson,  & Shojgreen-Downer,  1996;  Sizer,  1992;  Muncey  & McQuillan,  1996). 

Many  teachers  learned  to  teach  using  a model  of  teaching  and  learning  that  focuses  heavily  on 
memorizing  facts,  without  also  emphasizing  deeper  understanding  of  subject  knowledge  (Cohen, 
McLaughlin,  & Talbert,  1993;  Darling-Hammond  & McLaughlin,  1995;  Porter  & Brophy,  1988). 
Even  when  they  agree  that  new  teaching  approaches  are  needed,  teachers  often  are  unaware  that  their 
own  knowledge  and  skills,  or  teaching  practices,  are  inconsistent  with  high  standards,  or  that  they 
may  not  have  received  enough  preparation  to  implement  new  approaches  effectively. 

The  continual  deepening  of  knowledge  and  skills  is  an  integral  part  of  any  profession. 
Teaching  is  no  exception  (Shulman  & Sparks,  1992;  National  Board  for  Professional  Teaching 
Standards,  1989).  In  the  context  of  the  ambitious  education  reforms  being  undertaken  across  the 
nation,  the  quality  of  teachers’  professional  development  has  come  under  increased  scrutiny.  A large 
body  of  literature  has  emerged  focusing  on  what  effective  professional  development  for  teachers 
“looks”  like.  The  literature  suggests  that  traditional  approaches  to  professional  development,  such  as 
short  workshops  or  attendance  at  conferences,  may  foster  teachers’  awareness  or  interest  in 
deepening  their  knowledge  and  skills.  However,  such  approaches  to  professional  development 
appear  insufficient  to  foster  learning  that  would  fundamentally  alter  what  teachers  teach  or  how  they 
teach  it.  Yet,  for  many  of  the  nation’s  teachers,  professional  development  continues  to  be 
characterized  by  fragmented,  “one-shot”  workshops  at  which  teachers  listen  passively  to  “experts” 
and  learn  about  topics  that  are  not  essential  to  teaching  (National  Foundation  for  the  Improvement  of 
Education,  1996).  A national  survey  conducted  in  1998  reports  that,  depending  on  the  subject  of  the 
professional  development  activity,  between  44  and  81  percent  of  teachers  reported  that  professional 
development  activities  lasted  a total  of  one  to  eight  hours  during  the  previous  12  months  (U.S. 
Department  of  Education,  1999a).  Even  when  the  professional  development  focused  on  “in-depth 
study  in  the  subject  area”  of  teachers’  main  teaching  assignment,  only  56  percent  of  teachers 
reported  more  than  eight  hours  of  professional  development.  Further,  a study  of  professional 
development  in  NSF’s  statewide  systemic  initiatives  found  that  activities  were  often  of  insufficient 
duration  (Corcoran,  Shields,  & Zucker,  1998). 

Professional  development  that  has  a substantial  number  of  contact  hours  and  is  sustained 
over  a long  period  of  time  is  thought  to  have  a stronger  impact  on  teaching  practice  and  to  be  more 
consistent  with  systemic  reform  efforts  than  professional  development  of  limited  duration  (Corcoran, 
1995;  Darling-Hammond,  1995;  Hargreaves  & Fullan,  1992;  Hiebert,  1999;  Lieberman,  1996;  Little, 
1993;  Richardson,  1994;  Sparks  & Loucks-Horsley  1989;  Stiles,  Loucks-Horsley,  & Hewson,  1996). 
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Conventional  wisdom  suggests  that  certain  types  of  professional  development  activities  are  more 
likely  than  others  to  offer  such  sustained  learning  opportunities.  These  approaches  include:  study 
groups,  in  which  teachers  are  engaged  in  regular,  structured,  and  collaborative  interactions  around 
topics  identified  by  the  group;  coaching  or  mentoring  arrangements,  where  teachers  work  one-on-one 
with  an  equally  or  more  experienced  teacher;  networks,  which  link  teachers  or  groups,  either  in 
person  or  electronically,  to  explore  and  discuss  topics  of  intent,  pursue  common  goals,  share 
information,  and  address  common  concerns;  and  immersion  in  inquiry,  where  teachers  engage  in  the 
kinds  of  learning  that  they  are  expected  to  practice  with  their  students  (Loucks-Horsley  et  al.,  1998). 
Compared  to  workshops,  these  types  of  activities  are  typically  thought  to  be  longer,  to  allow  teachers 
the  opportunity  to  practice  and  reflect  upon  their  teaching,  and  to  be  more  embedded  in  ongoing 
teaching  practices.  A 1998  national  survey  found  that  many  teachers  believe  that  job-embedded, 
collaborative  professional  development  activities,  such  as  common  planning  time,  being  formally 
mentored  by  another  teacher,  or  networking  with  other  teachers  outside  the  school,  are  more  helpful 
than  traditional  forms  of  professional  development  (U.S.  Department  of  Education,  1999a). 

Although  there  is  a large  literature  on  professional  development,  little  high-quality  research 
has  been  conducted  on  the  relationship  between  characteristics  of  professional  development  and 
change  in  teachers’  classroom  teaching  practice,  and  still  less  has  been  conducted  on  the  relationship 
between  characteristics  of  professional  development  and  gains  in  student  achievement.  The  limited 
evidence  available  suggests  that  the  most  important  aspect  of  high-quality  professional  development 
activities  is  the  degree  to  which  they  focus  on  the  content  that  teachers  must  teach.  Two  recent 
studies  of  professional  development — a research  synthesis  of  professional  development  in 
mathematics  and  science  commissioned  for  this  evaluation,  as  well  as  a study  of  professional 
development  and  student  mathematics  achievement  in  California — came  to  the  same  conclusion. 
These  reviews  found  that  professional  development  focusing  on  subject  matter  content  and  how 
children  learn  that  content  is  effective  in  boosting  student  achievement.  Kennedy  (1998)  found  that 
professional  development  for  teachers  that  focuses  on  how  their  students  learn  a particular 
mathematical  or  scientific  idea  was  more  effective  in  boosting  these  students’  achievement  than 
professional  development  that  focused  on  general  principles  that  apply  to  teaching  all  mathematics 
or  science.  Cohen  and  Hill  (1998)  found  that  mathematics  teachers  who  participated  in  ongoing 
curriculum-centered  professional  development  were  more  likely  to  report  reform-oriented  teaching 
practices  in  mathematics  than  teachers  who  did  not  participate  in  such  professional  development,  and 
that  such  practices  were  associated  with  larger  schoolwide  gains  in  students’  mathematics 
performance. 

The  need  for  high-quality  professional  development  that  focuses  on  content  and  how  students 
learn  content  is  all  the  more  pressing  in  light  of  the  large  number  of  teachers  who  teach  outside  of 
their  areas  of  specialization  (National  Commission  on  Teaching  and  America’s  Future,  1996).  In 
1998,  12  percent  of  science  teachers  of  students  in  grades  7-12,  and  18  percent  of  mathematics 
teachers  in  these  grades,  had  neither  a major  nor  a minor  in  their  main  teaching  assignment  (U.S. 
Department  of  Education,  1999a).  The  severity  of  this  problem  varies  by  state,  with  some  states 
having  very  high  percentages  of  teachers  teaching  outside  their  areas  of  specialization  (Ingersoll, 
1996;  U.S.  Department  of  Education,  1999a).6  This  situation  is  especially  true  of  teachers  who  teach 
at-risk  students  and  those  who  teach  in  high-poverty  schools.  In  1998,  teachers  lacking  a major  in 
their  primary  assignment  taught  almost  a third  of  the  mathematics  classes  in  high-poverty  schools, 


6 While  teaching  out-of-field  is  clearly  an  important  issue,  most  statistics  on  out-of-field  teaching  do  not  consider  the 
proportion  of  time  spent  teaching  out-of-field,  and  thus  may  overstate  the  problem. 
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compared  to  14  percent  of  classes  in  low-poverty  schools  (U.S.  Department  of  Education,  1999a). 
Teachers  who  have  not  specialized  in  the  subjects  that  they  teach  must  often  teach  unfamiliar 
content.  Common  sense  and  research  make  clear  that  this  is  a recipe  for  disaster. 

The  Eisenhower  Professional  Development  Program  was  designed  to  help  address  all  of 
these  issues.  The  program’s  emphasis  is  to  support  professional  development  designed  to  help 
teachers  meet  the  new  demands  of  teaching  to  high  standards.  Yet,  prior  to  the  1994  reauthorization, 
program-funded  activities  were  not  generally  designed  to  provide  the  types  of  ongoing,  in-depth 
learning  opportunities  likely  to  deepen  teachers’  knowledge  and  skills,  or  change  classroom  practice. 
The  last  evaluation  of  the  Eisenhower  program,  conducted  in  1988-89,  described  it  as  a “modest 
investment”  that  maximizes  breadth  of  coverage  across  a large  number  of  teachers  rather  than  depth 
of  professional  development  (Knapp,  Zucker,  Adelman,  & St.  John,  1991).  That  evaluation 
indicated  that  district-supported  activities,  which  account  for  the  vast  majority  of  program  funds, 
tended  to  be  one-time  in-service  training  events,  averaging  six  hours  in  length.  In  fact,  in  one  quarter 
of  the  nation’s  districts,  typical  activities  lasted  less  than  four  hours.  Not  surprisingly,  therefore,  the 
evaluation  found  few  instances  of  professional  development  that  focused  on  teachers’  content 
knowledge  during  nearly  40  site  visits  to  school  districts  in  1988-89.  The  pattern  of  using  district 
Eisenhower  funds  for  short-term  professional  development  activities  was  recently  confirmed  by  an 
analysis  of  program  performance  reports;  in  1994-95,  54  percent  of  Eisenhower-assisted  district 
activities  provided  six  hours  or  less  of  professional  development  per  participant  (Donly  & Gutmann, 
1997).  By  contrast,  Eisenhower  activities  sponsored  by  SAHE  grantees  in  1988-89  typically  were 
more  intensive,  averaging  about  60  hours  per  participating  teacher.  These  SAHE-sponsored 
activities  also  were  more  likely  to  focus  on  mathematics  and  science  content  (Knapp  et  al.,  1991). 
The  1994  reauthorization  of  the  Eisenhower  Professional  Development  Program  represents  a strong 
effort  to  move  all  program-funded  activities  toward  sustained,  intensive,  high-quality  professional 
development  that  supports  high  academic  standards,  with  a special  emphasis  on  teachers  of  students 
in  high-poverty  schools. 


The  Eisenhower  Program  and  the  Goals  of  the  1994 

Reauthorization 


The  Eisenhower  Professional  Development  Program  was  established  in  1984,  and  was 
reauthorized  in  1988  and  again  in  1994  as  Title  II  of  the  Elementary  and  Secondary  Education  Act, 
as  amended  by  the  Improving  America’s  Schools  Act  (IASA)  of  1994.  The  Eisenhower  Professional 
Development  Program  allocates  funds  through  states  to  school  districts  and  to  institutions  of  higher 
education  and  nonprofit  organizations.  In  fiscal  year  1999,  $335  million  was  appropriated  for  Part  B 
of  the  Eisenhower  Professional  Development  Program,  State  and  Local  Activities.  Eisenhower  funds 
are  distributed  to  states  according  to  a formula  that  weights  equally  the  number  of  children  in  the 
state  between  the  ages  of  5 and  1 7 and  the  state’s  allocation  under  Title  I,  Part  A of  the  Elementary 
and  Secondary  Education  Act.7  Eighty-four  percent  of  allocated  Title  II,  Part  B funds  are  distributed 
to  SEAs,  with  the  remaining  16  percent  allocated  to  SAHEs.  At  least  90  percent  of  SEA  allocations 
then  flow  through  to  local  education  agencies  (LEAs),  based  on  the  same  formula  (equal  weights  to 


7 Part  B allocates  funds  to  the  50  states,  the  District  of  Columbia,  Puerto  Rico,  the  Bureau  of  Indian  Affairs  (BIA), 
and  the  outlying  areas. 
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the  school-aged  population  and  the  LEA’s  Title  I,  Part  A allocation).8  LEAs  that  receive  Eisenhower 
grants  under  $10,000  are  required  to  form  consortia  with  other  such  LEAs,  unless  the  SEA  waives 
the  requirement  (Section  2204(b)).  SAHEs  distribute  at  least  95  percent  of  their  Eisenhower 
allocations  by  competitive  grants  or  contracts  to  institutions  of  higher  education  (IHEs)  or  nonprofit 
organizations  (NPOs)  that  provide  professional  development  to  teachers  or  prospective  teachers. 

Each  SAHE  develops  priorities  and  guidelines  for  the  awards  based  on  the  state  plan  for 
improvement  in  teaching  and  learning,  which  it  develops  collaboratively  with  the  SEA  (Section 
2205(a)(2)(A)). 

Like  its  predecessor,  the  Eisenhower  Mathematics  and  Science  Education  Program,  the 
reauthorized  Eisenhower  Professional  Development  Program  focuses  on  the  professional 
development  of  mathematics  and  science  teachers.  The  reauthorized  legislation,  however,  expands 
the  program  by  allowing  states  and  districts  to  use  funds  in  excess  of  $250  million  to  provide 
professional  development  to  teachers  in  other  core  academic  subject  areas  (Section  2206).9 

In  its  1994  reauthorization  of  the  program,  Congress  made  it  explicit  that  Eisenhower- 
assisted  activities  should  be  designed  to  improve  teacher  practice  and,  ultimately,  student 
performance.  The  law  also  places  particular  emphasis  on  serving  teachers  in  schools  with  high 
poverty  rates.  Furthermore,  the  reauthorized  Eisenhower  program  embodies  policymakers’  intention 
that  the  program  support  systemic  education  reform  and  deeper  learning  among  teachers.  The  law 
incorporates  a number  of  strategies  to  achieve  these  goals. 

First,  and  most  important,  the  Eisenhower  program  is  intended  to  support  high-quality 
professional  development  activities.  Before  the  1994  reauthorization,  Congress  provided  limited 
direction  about  what  characterized  high-quality  professional  development;  currently,  both  the 
Eisenhower  Professional  Development  Program  legislation  and  the  program  guidance  published  by 
the  U.S.  Department  of  Education  (ED)  emphasize  that  the  Eisenhower  program  should  fund 
professional  development  that  is  sustained,  intensive,  ongoing,  and  of  high  quality.  Such 
professional  development  should  reflect  recent  research  on  teaching  and  learning  and  should  provide 
teachers  and  other  school  staff  with  the  knowledge  and  skills  necessary  to  provide  all  students  with 
the  opportunity  to  meet  challenging  standards  (Section  2002(2)).  Further,  these  provisions  are 
reflected  in  ED’s  performance  indicators  for  the  Eisenhower  program,  which  fulfill  one  of  ED’s 
responsibilities  under  the  Government  Performance  and  Results  Act  (GPRA).10 

Second,  the  Eisenhower  program  is  intended  to  ensure  that  professional  development 
activities  supported  with  Eisenhower  funds  include  and  target  teachers  of  at-risk  students. 

Reflecting  the  strong  emphasis  on  education  reform  efforts  and  on  federal  programs  to  increase 
access  to  a high-quality  education  for  all  students,  the  1994  legislation  requires  that  state  applications 
and  local  plans  must  take  into  account  the  educational  needs  of  students  from  historically 


8 Up  to  5 percent  of  the  SEA’s  Title  II  grant  may  be  used  for  program  administration,  and  another  5 percent  may  be 
used  to  support  professional  development  activities  provided  at  the  state  level. 

9 There  are  two  ways  that  Eisenhower  funds  can  be  used  to  support  professional  development  in  other  subject  areas. 
First,  when  the  appropriation  for  the  program  exceeds  $250  million,  the  additional  funds  can  be  used  to  provide 
professional  development  in  core  subject  areas  other  than  mathematics  and  science.  Second,  the  ESEA  legislation 
allows  states  and  districts  to  apply  to  the  federal  government  for  waivers  that  allow  them  to  devote  larger  percentages 
of  their  Eisenhower  Professional  Development  Program  grants  to  other  core  subject  areas. 

10  GPRA  requires  ED  to  establish  annual,  quantifiable  performance  goals  and  indicators  for  ED  programs  as  part  of  a 
strategic  planning  process. 
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underrepresented  populations.1 1 The  Eisenhower  legislation  also  places  special  emphasis  on 
addressing  the  needs  of  teachers  in  schools  receiving  Title  I,  Part  A funds;  generally  these  are 
schools  that  have  higher  rates  of  poverty  than  other  schools  in  their  districts.  Teachers  in  schools 
that  receive  Title  I support  must  be  involved  in  the  assessment  of  local  needs  for  professional 
development,  which  is  required  under  Title  II  (Section  2208(b)(2)).  Furthermore,  in  planning  for 
professional  development,  SEAs  and  LEAs  must  take  into  account  how  Title  II-funded  activities 
address  the  needs  of  teachers  in  schools  that  receive  Title  I support  (Section  2205(b)(2)(E)  and 
Section  2208(d)(1)(B)).  The  Title  I statute  requires  similar  coordination  with  the  Eisenhower 
Professional  Development  Program.  See  Section  1 1 1 9(b)(  1 1)(C). 

Third,  the  Eisenhower  program  is  intended  to  integrate  Eisenhower-assisted  activities  with 
other  reform  efforts.  Recent  efforts  to  improve  education  have  focused  on  ensuring  that  all  aspects 
of  the  education  system — including  curricula,  assessments,  teacher  education — be  consistent  with 
one  another  and  be  geared  toward  the  same  goals.  Reflecting  this  focus,  the  law  requires  the 
alignment  of  Eisenhower-assisted  professional  development  activities  with  challenging  state  and 
local  standards  and  coordination  of  supported  activities  with  education  reform  and  professional 
development  efforts  funded  by  federal,  state,  and  local  governments  and  other  public,  private,  and 
nonprofit  organizations  and  associations.  Such  integration  of  Eisenhower-assisted  activities  with 
other  reform  efforts  would  presumably  strengthen  the  quality  of  those  activities  by  gearing  them  to 
challenging  standards  and  by  allowing  several  funding  sources  to  be  combined  to  design  higher 
quality  activities.  The  law’s  requirements  for  district  planning  of  professional  development 
activities,  for  co-funding  those  activities  with  funds  from  other  programs,  and  for  IHE/NPOs 
working  with  schools,  school  districts,  or  consortia  of  districts,  all  promote  linkages  between 
Eisenhower-assisted  activities  and  those  funded  from  other  sources.  (See  Sections  2205(c), 
2208(d)(1)(H),  2209  (a),  and  221 1(a).) 

Finally,  the  reauthorized  Eisenhower  program  contains  provisions  intended  to  foster 
purposeful  planning  and  ongoing  tracking  of  progress  by  states  and  localities,  supported  by  state  and 
district  performance  indicators.  A number  of  the  law’s  requirements  encourage  SEAs  and  LEAs  to 
engage  in  a continuous  improvement  process,  grounded  in  careful  goal-setting  and  in  monitoring 
progress.  The  1994  law  establishes  detailed  requirements  for  state  and  local  planning  under  the  Title 
II  program.  States  and  localities  receiving  Title  II  funds  must  develop  plans  to  improve  teaching  and 
learning.  These  plans  must  be  based  on  needs  assessments,  must  be  developed  through  a 


11  The  legislation  requires  state  applications  and  local  plans  to  take  into  account  the  need  for  greater  access 
to  education  in  core  academic  subjects,  especially  mathematics  and  science,  “by  students  from  historically 
underrepresented  groups,  including  females,  minorities,  individuals  with  limited  English  proficiency,  the 
economically  disadvantaged,  and  individuals  with  disabilities,  by  incorporating  pedagogical  strategies  and 
techniques  which  meet  such  individuals’  educational  needs”  (Section  2205(b)(2)(F)).  Similar  language  is 
also  used  to  describe  how  the  LEA  will  use  needs  assessment  to  plan  professional  development  activities 
that  address  the  needs  of  diverse  student  populations  (Section  2208  (d)(1)(F)). 
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participatory  process,  and  must  describe  how  the  strategy  for  professional  development  will  meet 
identified  needs  (Sections  2205(b)(2),  2208(c)(2),  and  2208(d)(1)).12 

An  important  aspect  of  local  planning  is  the  participation  of  teachers  in  decisions  about  the 
use  of  Eisenhower  Professional  Development  Program  funds.  The  reauthorization  legislation 
specifically  states  that  LEAs 

shall  use  not  less  than  80  percent  of  such  [Eisenhower  Professional  Development 
Program]  funds  for  professional  development  of  teachers,  and,  where  appropriate, 
administrators,  and,  where  appropriate,  pupil  services  personnel,  parents,  and  other 
staff  of  individual  schools  in  a manner  that  (A)  is  determined  by  such  teachers  and 
staff;  [and]  (B)  to  the  extent  practicable,  takes  place  at  the  individual  school  site 
(Section  2210(a)(1)). 

This  provision  reflects  the  Congress’  conclusion  that  decisions  about  professional  development  are 
best  made  by  its  participants.  At  the  same  time,  the  law  also  states  that  this  professional 
development  should  be  consistent  with  the  LEA’s  overall  plan  for  professional  development  (Section 
2210(a)(1)(C)). 

Some  of  the  key  goals  of  the  Eisenhower  program  are  summarized  in  a set  of  performance 
indicators  prepared  by  ED,  as  required  by  the  Government  Performance  and  Results  Act  (GPRA). 
GPRA  requires  ED  to  establish  annual,  quantifiable  performance  goals  and  indicators  for  ED 
programs  as  part  of  a strategic  planning  process.  The  current  indicators  for  Part  B of  the  Eisenhower 
program  reflect  the  intent  of  the  1994  legislation  (U.S.  Department  of  Education,  1999c): 

Objective  1 Classroom  instruction  is  improved  through  effective  professional  development. 

Indicator  1.1  Teachers'  skills  and  classroom  instruction.  By  1998,  over  50 
percent  of  a sample  of  teachers  will  show  evidence  that  participation  in  Eisenhower- 
assisted  professional  development  has  resulted  in  an  improvement  in  their  knowledge 
and  skills,  and  by  2000,  over  60  percent  will  show  such  evidence.  By  1999,  over  50 
percent  of  a sample  of  teachers  in  selected  sites  will  show  evidence  that  participation 
in  Eisenhower-assisted  professional  development  has  resulted  in  an  improvement  in 
classroom  instruction. 


12  When  applying  for  Title  II  funds,  states  may  elect  either  to  submit  a program-specific  application  to  ED  or  to 
include  Title  II  in  an  ESEA  “consolidated  application.”  If  the  state  submits  a Title  II-specific  application,  it  must 
include  statutorily  required  information  about  the  needs  assessment  it  has  conducted  and  its  plan  to  improve  teaching 
and  learning.  (See,  generally,  section  14302.)  If  the  state  submits  a consolidated  application,  it  need  not  include  this 
information  in  its  application.  Similarly,  states  can  require  LEAs  to  submit  either  a Title  II  specific  or  consolidated 
application.  If  the  state  requires  a Title  II-specific  application,  it  must  include  statutorily  required  information  about 
its  plan  for  professional  development  and  its  needs  assessment.  If  the  state  requires  a consolidated  application,  it 
need  not  include  this  information  in  that  application  unless  the  state  requires  it  to  do  so.  Regardless  of  the  content  of 
state-  or  LEA-consolidated  applications,  ED  has  made  it  clear  to  states  and  districts  that,  if  they  include  the 
Eisenhower  Professional  Development  Program  in  consolidated  applications,  they  still  must  implement  all  planning 
requirements  that  apply  to  the  program.  However,  information  about  planning  and  needs  assessments  need  not  be 
included  in  the  consolidated  application  itself  or  otherwise  prepared  in  the  formal  planning  document. 
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Objective  2 


Objective  3 


Objective  4 


Objective  5 


O 


High-quality  professional  development  and  state  policy  are  aligned  with  high 
state  content  and  student  performance  standards. 

Indicator  2.1  District-level  professional  development.  By  1998,  over  50  percent 
of  teachers  participating  in  district-level  or  higher  education  Eisenhower-assisted 
professional  development  will  participate  in  activities  that  are  aligned  with  high 
standards.  By  2000,  over  75  percent  will. 

Professional  development  is  sustained,  intensive,  and  high  quality  and  has  a 
lasting  impact  on  classroom  instruction. 

Indicator  3.1  High  quality.  By  1998,  over  50  percent  of  teachers  participating  in 
district-level,  Eisenhower-assisted  professional  development  activities  will 
participate  in  activities  reflecting  best  practices,  including  a focus  on  continuous 
improvement.  By  2000,  over  75  percent  will. 

Indicator  3.2  Sustained  professional  development.  By  1998,  35  percent  of 
teachers  participating  in  district-level,  Eisenhower-assisted  activities  will  participate 
in  activities  that  are  a component  of  professional  development  that  extends  over  the 
school  year.  By  2000,  over  50  percent  will. 

High-quality  professional  development  is  provided  to  teachers  who  work  with 
disadvantaged  populations. 

Indicator  4.1  High-poverty  schools.  The  proportion  of  teachers  participating  in 
Eisenhower-assisted  activities  who  teach  in  high-poverty  schools  will  exceed  the 
proportion  of  the  national  teacher  pool  who  teach  in  high-poverty  schools. 

Indicator  4.2  Context  (not  limited  to  any  single  program):  Teachers.  Teachers 
in  high-poverty  schools  will  participate  in  intensive,  sustained,  high-quality 
professional  development  at  rates  comparable  to  or  higher  than  the  rates  for  teachers 
in  other  schools. 

Effective  management  of  the  Eisenhower  Program  at  the  federal,  state,  and  local 
levels  supports  systemic  reform. 

Indicator  5.1  Federal  guidance  and  assistance.  The  number  of  Eisenhower  state 
coordinators  who  report  that  ED  guidance  and  assistance  are  timely  and  helpful  will 
increase. 
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Objective  6 Measurement  of  integrated  planning  and  collaboration. 

Indicator  6.1  By  1998,  50  percent  of  all  states  will  have  developed  performance 
indicators  for  integrated  professional  development  across  programs  (including 
Eisenhower)  in  order  to  support  systemic  reform  and  will  have  data-collection 
systems  in  place.  By  2000,  75  percent  will  have  these  indicators. 

Indicator  6.2  By  2000,  over  80  percent  of  states  will  report  that  they  coordinate  and 
collaborate  with  Title  I State  coordinators  when  they  develop  their  plans  for 
professional  development. 

This  evaluation  of  the  Eisenhower  program  addresses  Objectives  1 through  4.  This  report  is 
designed  to  assist  policymakers  in  determining  whether  and  how  program-funded  activities 
contribute  to  the  nation’s  efforts  to  improve  schools,  and  to  lay  the  foundation  for  deliberations  about 
the  Eisenhower  program  during  the  current  reauthorization  of  ESEA.  The  next  section  describes  the 
types  of  information  collected  in  the  evaluation  in  order  to  provide  a comprehensive  and  accurate 
picture  of  the  Eisenhower  program. 


Approach  to  Evaluation  of  the  Eisenhower  Program 


In  February  1997,  the  U.S.  Department  of  Education’s  Planning  and  Evaluation  Service 
commissioned  the  American  Institutes  for  Research  (AIR)  to  conduct  a three-year  evaluation  of  Part 
B of  the  Eisenhower  Professional  Development  Program,  Title  II  of  the  Elementary  and  Secondary 
Education  Act,  as  amended  by  the  Improving  America’s  Schools  Act  (IASA).  The  evaluation  had 
two  purposes: 

1.  To  describe  Eisenhower-assisted  activities  and  evaluate  their  effects.  The  1994 
reauthorization  of  the  program  instituted  a number  of  far-reaching  changes  described 
above.  ED  wanted  AIR  to  describe  program-funded  activities  and  to  obtain  an 
understanding  of  whether  the  changes  were  occurring. 

2.  To  provide  information  related  to  performance  indicators  that  ED  developed  for  the 
program  in  response  to  requirements  of  the  Government  Performance  and  Results  Act 
(GPRA).  GPRA  requires  ED  to  determine  the  program’s  performance  in  relation  to  its 
goals  and  objectives. 

To  meet  ED’s  information  needs,  the  evaluation  team  designed  an  integrated  set  of  data 
collection  activities.  The  evaluation  used  a multi-method  strategy  to  collect  quantitative  and 
qualitative  data  about  Eisenhower-assisted  activities.  These  data  come  from  a variety  of  sources — 
state  and  district  officials,  directors  of  grants  awarded  to  institutions  of  higher  education  and 
nonprofit  organizations,  and  teachers.  The  evaluation  was  designed  to  obtain  national  data  about 
program-funded  activities,  to  obtain  a deeper  understanding  of  how  the  program  works  in  selected 
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locations,  and  to  collect  information  about  how  professional  development  activities  affect  teacher 
practice.13  Exhibit  1.0  displays  a timeline  of  the  evaluation’s  major  activities. 

As  Exhibit  1.0  shows,  the  evaluation  is  based  on  three  strands  of  data  collection.  The  first 
strand,  a National  Profile , provides  information  about  program  goals,  strategies,  operations,  and 
activities  nationwide.  During  the  1997-98  school  year,  we  conducted  telephone  interviews  with  a 
national  probability  sample  of  Eisenhower  coordinators  in  363  school  districts  and  SAHE-grantee 
project  directors  in  92  institutions  of  higher  education  or  nonprofit  organizations  (IHE/NPOs).14  We 
also  collected  data  from  a mail  survey  of  a national  probability  sample  of  1,027  teachers  who 
participated  in  657  Eisenhower-assisted  activities.15  These  Teacher  Activity  Survey  data  describe 
the  types  of  professional  development  supported  with  Eisenhower  funds,  and  compare  activities 
sponsored  by  school  districts  to  those  sponsored  by  higher  education  institutions  and  nonprofits. 
Appendix  A provides  additional  detail  about  the  sampling  design  and  methodology  of  the  three 
components  of  the  National  Profile 

A second  strand  of  data,  a set  of  Case  Studies , provides  detailed  information  about  how  the 
Eisenhower  Professional  Development  Program  operates  in  selected  states,  school  districts,  and 
schools.  During  the  1997-98  school  year,  AIR  conducted  In-Depth  Case  Studies  in  10  school 
districts — two  school  districts  in  each  of  five  states:  Kentucky,  New  York,  Ohio,  Texas,  and 
Washington.  The  districts  were  selected  to  represent  a diversity  of  region,  urbanicity,  poverty  level, 
and  ethnic  composition.  The  sites  also  were  selected  because  they  supported  diverse  approaches  to 
professional  development,  instead  of,  or  in  addition  to,  traditional  workshops  or  conference 
attendance.  For  example,  a number  of  the  in-depth  case  districts  used  their  Eisenhower  funds  to 
provide  professional  development  to  “lead”  or  “resource”  teachers  who  became  mentors  to  other 
teachers  in  their  schools  or  districts.  Other  in-depth  case  study  districts  used  Eisenhower  funds  to 
support  workshops  or  institutes  that  were  unusual  because  they  extended  over  many  days  or  weeks. 
Still  other  districts  were  selected  because  they  supported  school-based  professional  development 
activities  with  Eisenhower  funds.  Through  site  visits  to  the  in-depth  case  study  districts,  we  explored 
how  decisions  are  made  about  the  use  of  program  funds,  and  the  reasons  that  the  goals,  operations, 
and  activities  vary  across  states  and  districts.  The  case  studies  have  been  a critical  source  of 
information  about  how  Eisenhower-assisted  activities  relate  to  other  professional  development  and 
education  reform  efforts,  and  about  the  degree  of  coherence  and  consistency  among  these  efforts. 


13  The  evaluation  was  designed  to  collect  data  about  six  research  questions:  1 ) What  types  of  professional 
development  activities  does  the  Eisenhower  program  make  available  to  teachers,  and  to  what  extent  do  these 
activities  represent  best  practices?  2)  Who  participates  in  Eisenhower-assisted  professional  development  activities? 
3)  As  designed,  planned,  and  implemented  at  the  state,  district,  and  school  levels,  how  does  the  Eisenhower  program 
fit  into  the  mosaic  of  professional  development  and  other  systemic  reform  activities?  4)  From  the  teacher’s 
perspective,  how  do  Eisenhower-assisted  and  other  professional  development  activities  combine  to  provide  a 
coherent,  integrated  set  of  learning  opportunities?  5)  Do  teachers’  experiences  in  Eisenhower-assisted  professional 
development  activities,  in  the  context  of  other  professional  development  activities,  contribute  to  teaching  practice? 
and  6)  How  is  the  Eisenhower  program  planned,  coordinated,  and  evaluated  at  the  state,  district,  and  IHE  levels? 
While  these  research  questions  guided  the  evaluation,  they  do  not  serve  as  the  framework  for  reporting  findings  in 
this  report.  Rather,  we  report  analyses  from  each  of  our  data  sets  in  turn,  as  depicted  in  Exhibit  1.3. 

14  The  telephone  interviews  of  363  district  coordinators  represent  a response  rate  of  88  percent  of  sampled 
coordinators;  the  telephone  interviews  with  IHE/NPO  project  directors  represent  a response  rate  of  87  percent  of 
sampled  project  directors.  Details  regarding  sampling  are  described  in  Appendix  A. 

15  The  mail  survey  of  teachers  represents  a response  rate  of  72  percent  of  sampled  teachers.  Details  regarding 
sampling  are  provided  in  Appendix  A. 
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The  information  from  the  in-depth  case  studies  expands  upon  information  we  obtained  from 
Exploratory  Case  Studies  during  the  1 996-97  school  year.  Like  the  in-depth  cases,  the  exploratory  case 
studies,  conducted  in  six  school  districts,  were  selected  to  vary  by  region,  urbanicity,  poverty  level,  and 
ethnic  composition.16  The  first  report  from  this  evaluation,  the  Eisenhower  Professional  Development 
Program:  Emerging  Themes  from  Six  Districts  (Birman,  Reeve,  & Sattler,  1998),  was  based  on  these 
exploratory  case  studies,  and  we  also  use  information  from  the  exploratory  cases  in  this  report.  Exhibits 
1 . 1 and  1 .2  provide  pseudonyms  and  demographic  information  about  the  in-depth  and  exploratory  case 
studies.  More  detail  about  site  selection,  design,  and  methodology  of  the  case  studies,  including  their 
approaches  to  professional  development,  is  contained  in  Appendix  B. 

The  third  strand  of  this  evaluation,  a Longitudinal  Study  of  Teacher  Change,  examines  the  effects 
of  Eisenhower-assisted  and  other  professional  development  on  teacher  practice  in  mathematics  and 
science.  For  the  in-depth  case  studies,  we  interviewed  and  conducted  classroom  observations  of  teachers 
in  three  schools — an  elementary,  middle,  and  high  school — in  each  of  the  10  districts.  We  also  surveyed 
all  teachers  who  teach  mathematics  or  science  in  those  schools.  We  asked  these  teachers  detailed 
questions  about  the  topics  they  covered,  their  goals  for  student  performance,  and  their  participation  in 
professional  development  activities  at  three  points  in  time:  the  fall  of  1997,  the  spring  of  1998,  and  the 
spring  of  1999.  The  three  waves  of  the  survey  provide  data  pertaining  to  the  1996-97,  1997-98,  and  1998- 
99  school  years.  The  Longitudinal  Study  of  Teacher  Change  will  enable  us  to  examine  the  extent  to  which 
teachers’  participation  in  Eisenhower-assisted  and  other  professional  development  activities  improves 
instruction  over  time.  Appendix  C provides  additional  information  about  the  sampling,  design,  and 
methodology  of  the  Longitudinal  Study  of  Teacher  Change. 

The  three  strands  of  the  evaluation  were  designed  to  produce  an  integrated  portrait  of  the 
Eisenhower  program.  The  study  does  not  describe  the  program  simply  from  the  perspective  of  its 
administrators,  but  also  from  the  perspective  of  the  teachers  who  participate  in  program-funded  activities. 
The  study  supplements  self-reported  information  from  teachers  and  administrators  with  rich  contextual 
information  from  case  studies.  Finally,  the  study  does  not  rely  only  on  teacher  reports  of  changes  in  their 
classroom  practice,  but  also  is  collecting  data  on  teaching  practice  at  three  points  in  time.  Thus,  because 
the  evaluation  involves  a variety  of  research  methods  and  collects  data  from  groups  of  individuals  who 
view  Eisenhower-assisted  activities  from  different  vantage  points,  it  is  able  to  provide  an  accurate 
description  of  program-funded  activities  and  analyses  of  the  features  of  these  activities  and  their  effects  on 
teacher  practice. 


16  In  addition  to  being  selected  for  their  demographic  features,  the  six  exploratory  sites  also  represented  districts  with 
features  that  we  believed  might  influence  the  program’s  implementation.  The  six  sites  were  selected  to  include:  one  district 
that  participated  in  the  Eisenhower  program  through  a consortium;  at  least  two  districts  that  had  IHE-supported  projects 
working  in  schools  in  the  district;  and  two  districts  located  in  states  that  received  waivers  from  ED  allowing  greater 
proportions  of  Eisenhower  funds  to  support  professional  development  in  areas  outside  mathematics  and  science. 
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EXHIBIT  1.1 


In-Depth  Case  Studies 

District 

Pseudonym 

Characteristics 

Richmond 

An  urban  district  in  New  York  State,  Richmond  serves  approximately  10,000  students  from  a 
predominantly  minority  population.  Nearly  60  percent  of  the  students  are  African  American, 
and  half  of  the  students  qualify  for  a free  or  reduced-price  lunch. 

East  City 

An  urban  district  in  New  York  City,  East  City  serves  a largely  immigrant  population.  Between 
25  and  30  percent  of  its  children  are  taught  in  Spanish,  while  many  of  the  other  children  speak 
Creole,  African  languages,  or  French.  One  hundred  percent  of  the  students  qualify  for  a free  or 
reduced-price  lunch. 

Maple  City 

A large  metropolitan  school  district  in  Ohio,  Maple  City  serves  nearly  64,000  students.  Nearly 
55  percent  of  these  students  are  African  American,  and  60  percent  are  eligible  for  a free  or 
reduced-price  lunch. 

Buckeye 

A suburban  school  district  in  Ohio,  Buckeye  serves  approximately  1 1,000  children.  About  90 
percent  of  the  students  are  white.  A very  small  percentage  of  students  qualifies  for  a free  or 
reduced-price  lunch. 

Rhinestone 

A metropolitan  school  district  in  Texas,  Rhinestone  serves  approximately  13,000  students  from 
a predominantly  minority  population:  43  percent  of  the  students  are  Hispanic,  35  percent  are 
African  American,  and  20  percent  are  white.  Fifty-six  percent  of  the  students  qualify  for  a free 
or  reduced-price  lunch. 

Lone  Star 

In  a large  city  in  Texas,  Lone  Star  serves  approximately  65,000  students  from  a predominantly 
minority  population.  Seventy-seven  percent  of  students  are  Hispanic,  and  18  percent  are  non- 
Hispanic  whites.  Thirty-two  percent  of  the  students  have  limited  English  proficiency,  and  67 
percent  of  the  students  qualify  for  a free  or  reduced-price  lunch. 

Riverside 

In  an  urban  fringe  of  a large  city  in  Washington,  Riverside  serves  20,000  students  in  a 
relatively  affluent  community.  Eighty  percent  of  the  students  are  white,  and  15  to  20  percent 
of  the  students  qualify  for  a free  or  reduced-price  lunch. 

Rainforest 

A small  logging  community  in  Washington,  Rainforest  serves  1,570  students,  85  percent  of 
whom  are  white.  Ten  percent  of  the  students  are  Native  American,  and  nearly  half  of  the 
students  qualify  for  a free  or  reduced-price  lunch. 

Weller 

A poor  rural  district  in  Kentucky’s  Appalachian  Mountains,  Weller  serves  500  students. 
Ninety-nine  percent  of  the  students  are  white,  and  60  percent  qualify  for  a free  lunch. 

Boonetown 

An  urban  fringe  of  a mid-sized  city  in  Kentucky,  Boonetown  serves  8,000  students,  95  percent 
of  whom  are  white.  A very  small  percentage  of  students  qualifies  for  a free  or  reduced-price 
lunch. 

EXHIBIT  1.2 


Exploratory  Case  Districts 


District 

Pseudonym 

Characteristics 

West  City 

A large,  urban  district  on  the  west  coast,  West  City  serves  a predominantly  minority  population 
of  61,889  students;  fewer  than  15  percent  of  students  are  white,  while  nearly  half  are  Asian,  and 
another  20  percent  are  Hispanic.  About  half  of  the  district’s  students  qualify  for  a free  or 
reduced-price  lunch. 

Middle  City 

An  urban  district  in  the  midwest,  Middle  City  serves  nearly  100,000  students.  Nearly  60 
percent  of  these  students  are  African  American,  and  almost  two  thirds  qualify  for  a free  or 
reduced-price  school  lunch. 

South  City 

A large,  urban  district  in  the  southeast,  South  City  serves  predominantly  minority  students, 
many  of  whom  are  not  native  English  speakers.  Nearly  half  of  the  district’s  333,817  students 
qualify  for  a free  or  reduced-price  lunch. 

Commuteville 

A large,  suburban  county  school  district  in  the  mid-Atlantic  region,  Commuteville  serves  an 
ethnically  diverse  population.  Just  over  two  thirds  of  students  are  white,  with  almost  equal 
representation  of  African  American,  Hispanic,  and  Asian  students.  About  12  percent  of  the 
students  qualify  for  a free  or  reduced-price  lunch. 

Northtown 

Northtown  is  a small  city  in  New  England.  Its  population  is  predominantly  white  (about  80 
percent  of  students),  and  over  a third  of  students  qualify  for  a free  or  reduced-price  lunch. 

Countryplace 

Countryplace  is  a consortium  of  seven  school  districts  in  a rural  part  of  the  midwest.  The 
population  is  fairly  homogeneous;  virtually  all  of  the  6,000  students  served  in  the  consortium 
are  white,  and  less  than  20  percent  qualify  for  a free  or  reduced-price  lunch. 

Overview  of  This  Report 


This  report  presents  information  from  all  three  strands  of  the  evaluation.  Exhibit  1.3  displays  the 
logic  of  the  Eisenhower  program  as  it  is  laid  out  in  the  legislation.  Improving  teaching  practice  (Box  5)  is 
the  goal  of  the  Eisenhower  legislation.  Teacher  experiences  in  Eisenhower-assisted  professional 
development  activities  (Box  4)  are  intended  to  improve  teaching  practice.  The  quality  of  the  activities 
districts  and  SAHE  grantees  make  available,  and  the  ways  districts  and  SAHE  grantees  select  teachers  to 
participate  (Box  3),  in  turn  influence  teacher  experiences  in  Eisenhower-assisted  professional  development. 
We  hypothesize  that  the  overall  quality  of  Eisenhower-assisted  activities  is  shaped  by  the  degree  of 
integration  of  the  Eisenhower-assisted  activities  with  other  professional  development  and  systemic  reform 
efforts,  as  well  as  by  how  districts  and  SAHE  grantees  plan  and  evaluate  Eisenhower  funded  activities 
(Box  2). 

In  addition  to  displaying  the  logic  of  the  Eisenhower  program,  Exhibit  1.3  also  describes  the 
structure  of  this  report.  The  report  starts  by  focusing  on  classroom  teaching  practice  (Box  5),  then 
examines  the  professional  development  experiences  of  teachers,  including  their  Eisenhower-assisted 
activities  (Box  4).  The  report  then  describes  how  the  program  operates  in  school  districts  and  in  SAHE- 
supported  projects  in  IHEs  and  nonprofits  (Boxes  3 and  2).  Throughout  the  report,  we  pay  attention  to 
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contextual  features  that  affect  program-funded  activities,  such  as  district  size,  district  poverty  level,  and  the 
characteristics  of  SAHE-grantees  (Box  1 ). 


EXHIBIT  1.3 

Program  Logic  Model  for  This  Evaluation 


We  begin  this  report  with  a description  of  classroom  practice,  then  describe  the  characteristics  of 
the  professional  development  activities  that  teachers  participate  in,  and  finally  describe  the  characteristics 
of  Eisenhower-assisted  operations.  In  doing  so,  we  have  adopted  an  approach  that  is  unusual  for  a federal 
program  evaluation,  which  usually  starts  with  a description  of  the  “program”  and  its  characteristics.  We 
have  chosen  this  alternative  approach  because  it  is  important  to  understand  what  types  of  teacher  practice 
the  Eisenhower  program  is  attempting  to  reinforce  and  to  change,  and  because  it  allows  us  to  avoid  a “top- 
down”  perspective  on  federal  programs.  This  perspective,  common  among  evaluations  of  federal 
programs,  implies  that  federal  provisions  are  at  the  center  of  change  in  teacher  practice.  Yet,  the  myriad 
evaluations  of  federal  education  programs  over  the  years  illustrate  that  federal  education  programs  are  only 
one  aspect  of  the  many  influences  upon  practice. 

Chapter  2 begins  with  an  analysis  of  teaching  practice  because  the  Eisenhower  program’s  goal  is  to 
improve  instruction,  and  ultimately  student  performance.  The  chapter  examines  the  practice  of  teachers  in 
the  30  schools  that  comprise  our  Longitudinal  Study  of  Teacher  Change;  the  chapter  also  compares 
teachers’  reports  of  their  classroom  teaching  to  the  standards  embodied  in  the  National  Assessment  of 
Educational  Progress  (NAEP). 

The  model  on  which  the  Eisenhower  program  is  based  assumes  that  teachers’  experiences  in 
Eisenhower-assisted  professional  development  activities  will  enhance  their  practice.  Therefore,  Chapter  3 
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analyzes  the  characteristics  of  Eisenhower-assisted  professional  development  activities  using  data  from  the 
Teacher  Activity  Survey  of  our  National  Profile.  In  addition,  the  chapter  describes  the  features  of 
Eisenhower-assisted  activities,  the  characteristics  of  teachers  who  participate  in  them,  the  coherence  of 
professional  development  from  the  teachers’  perspectives,  and  teachers’  reports  of  the  impact  of 
professional  development  on  their  knowledge  and  skills  and  classroom  practice.  The  chapter  also 
compares  the  characteristics  of  Eisenhower-assisted  activities  supported  by  school  districts  to  those 
supported  by  SAHEs.  Finally,  the  chapter  identifies  several  features  of  effective  professional  development 
and  explores  how  the  features  of  professional  development  relate  to  one  another  and  to  teacher  outcomes. 

The  Eisenhower  legislation  relies  on  school  districts  and  SAHE  grantees  to  design  professional 
development  activities  and  to  recruit  and  select  teachers  for  these  activities.  Because  of  the  important  role 
that  school  districts  play  in  the  Eisenhower  legislation,  Chapter  4 uses  data  from  district  Eisenhower 
coordinators  and  data  from  our  in-depth  and  exploratory  case  studies  to  describe  district  “portfolios”  of 
Eisenhower-assisted  activities.  A district’s  portfolio  is  the  collection  of  Eisenhower-assisted  activities 
offered  in  a district;  the  activities  in  a district’s  portfolio,  taken  together,  represent  the  district’s  strategy  for 
addressing  the  goals  of  the  Eisenhower  legislation.  This  chapter  describes  the  characteristics  of  activities 
supported  by  Eisenhower  funds,  and  the  types  of  teachers  participating  in  these  activities. 

The  Eisenhower  legislation  also  lays  out  a variety  of  procedural  requirements  that  districts  must  use 
to  administer  the  Eisenhower  program.  Chapter  5 examines  the  ways  districts  link  Eisenhower-funded 
activities  with  those  funded  by  other  programs,  as  well  as  the  ways  they  plan  and  evaluate  supported 
activities.  In  particular,  the  chapter  examines  how  districts  build  a vision  for  professional  development  by 
aligning  Eisenhower- assisted  activities  with  state  and  district  standards  and  assessments,  and  by 
coordinating  these  activities  with  other  school  reform  and  professional  development  efforts.  The  chapter 
also  examines  how  districts  plan  and  evaluate  Eisenhower-assisted  activities,  including  their  use  of 
performance  indicators. 


SAHE  grantees  confront  some  issues  that  parallel  those  faced  by  the  districts.  Like  districts,  SAHE 
grantees  design  professional  development  activities  and  recruit  teachers  to  participate  in  them.  However, 
SAHE  grantees  in  other  ways  operate  quite  differently  from  school  districts.  SAHE  grantees  have  won 
competitions  within  their  states  to  conduct  particular  activities  rather  than  receiving  Eisenhower  funds  by 
formula  as  districts  do.  In  designing  the  grant  and  contract  competitions,  SAHEs  develop  priorities  and 
guidelines  for  professional  development  based  on  state  needs  assessments;  therefore  grantees  must  design 
their  activities  to  be  responsive  to  the  criteria  outlined  by  the  SAHE.  Unlike  districts,  however,  SAHE 
grantees  do  not  have  the  responsibility  to  address  the  professional  development  needs  of  all  teachers  in 
their  jurisdictions.  Using  data  from  the  National  Profile’s  telephone  interviews  with  SAHE-grantee  project 
directors,  Chapter  6 focuses  on  how  SAHE-grantees  operate  their  Eisenhower-assisted  projects. 


Eisenhower-assisted  activities,  in  both  districts  and  SAHE  grantees,  do  not  occur  in  a vacuum.  A 
district’s  characteristics,  such  as  size  and  poverty  rate,  may  affect  its  approach  to  professional  development 
activities.  For  example,  large  districts  have  more  access  to  other  funding  sources  for  professional 
development  than  smaller  districts  and,  therefore,  may  be  more  likely  to  coordinate  their  Eisenhower- 
assisted  activities  with  those  of  other  programs.  Districts  with  high  rates  of  poverty  might  be  more  likely 
to  target  teachers  of  at-risk  students.  SAHE  grantees  also  differ  from  one  another  in  a number  of  ways  that 
may  affect  the  characteristics  of  the  Eisenhower-assisted  activities  that  they  support.  For  example,  SAHE- 
grantee  project  directors  from  university  education  departments  are  likely  to  have  different  areas  of 
expertise  from  those  in  mathematics  departments,  and  project  directors  in  research  universities  might  have 
access  to  different  types  of  resources  than  those  in  other  types  of  institutions.  Box  1 of  Exhibit  1.3 
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represents  the  contextual  features  of  school  districts  and  SAHE  grantees  that  may  affect  how  program- 
funded  activities  operate.  Throughout  this  report,  we  examine  these  contextual  features  and  their  possible 
effects  on  districts,  institutions  of  higher  education,  and  nonprofits. 

The  organization  of  this  report  allows  the  reader  first  to  learn  about  teachers’  practices  in  the 
classroom,  then  the  Eisenhower-assisted  professional  development  activities  that  are  designed  to  affect 
their  practices,  and  finally,  district  and  SAHE-grantee  management  and  operation  of  these  activities.  In 
this  way,  we  hope  that  our  analysis  and  discussion  of  Eisenhower-assisted  activities  and  the  operation  of 
those  activities  will  be  grounded  in  a knowledge  and  understanding  of  the  teacher  classroom  practices  that 
the  Eisenhower  program  is  designed  to  improve. 
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CHAPTER  2 


TEACHING  PRACTICES  IN  MATHEMATICS  AND  SCIENCE 

The  primary  goal  of  the  Eisenhower  Professional  Development  Program  is  to  support 
professional  development  experiences  for  teachers  that  will  enhance  classroom  teaching  and, 
ultimately,  improve  student  learning.  Because  improved  teaching  is  critical  to  improved  learning  for 
students,  it  is  a cornerstone  of  the  standards  movement.  Therefore,  this  report  begins  by  focusing  on 
classroom  teaching  practice. 

The  purpose  of  this  chapter  is  to  lay  a foundation  for  our  evaluation  of  the  Eisenhower 
program,  by  reviewing  the  current  literature  on  teaching  and  learning  in  mathematics  and  science  and 
by  describing  classroom  teaching  practices  in  the  30  in-depth  case  study  schools.  By  drawing  on  the 
literature  and  data  on  the  30  case  schools,  we  will  characterize  the  strengths  and  weaknesses  of 
current  teaching  practices  in  mathematics  and  science  and  identify  areas  in  which  further  professional 
development  should  focus. 

The  data  on  classroom  practice  that  we  will  report  come  from  two  sources:  the  baseline  year 
of  our  three-year  longitudinal  survey  of  teacher  change  and  classroom  observations.  In  a subsequent 
report,  we  will  use  the  second  and  third  waves  of  the  longitudinal  survey,  along  with  the  data 
reported  here,  to  examine  the  effects  of  participation  in  professional  development  on  changes  over 
time  in  teaching  practice.  The  data  we  report  in  this  chapter  are  thus  the  first  step  in  our  examination 
of  the  effect  of  Eisenhower-assisted  activities  on  teaching  practice.  Exhibit  2.0  shows  how  the 
material  covered  in  this  chapter  fits  into  the  entire  framework  of  the  evaluation. 

EXHIBIT  2.0 


Conceptual  Framework  for  This  Evaluation 
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Data  Sources 


This  chapter  draws  on  two  sources  of  data  collected  for  the  Longitudinal  Study  of  Teacher 
Change:  1)  a baseline  survey  of  teachers  in  30  schools  and  2)  observations  of  classes  for  a sample  of 
teachers  (two  per  school)  who  completed  the  surveys.1  In  addition,  we  conducted  a content  analysis 
of  the  National  Assessment  of  Educational  Progress,  in  order  to  compare  the  content  of  instruction 
that  teachers  reported  in  the  survey  to  a national  standards  for  content.  In  the  paragraphs  below,  we 
describe  each  of  these  sources  in  turn. 

Baseline  Wave  of  the  Longitudinal  Teacher  Survey 

To  gather  data  on  teachers’  classroom  teaching  practices,  we  surveyed  all  teachers  who  teach 
mathematics  or  science  in  the  30  schools — one  elementary,  one  middle,  and  one  high  school  in  each 
of  the  10  in-depth  case  study  districts.  The  baseline  wave  of  the  survey,  which  was  conducted  in  the 
fall  of  1997,  asked  teachers  to  describe  their  teaching  during  the  1996-97  year.2  In  the  survey,  we 
asked  teachers  to  select  a year-long  mathematics  or  science  course  to  describe.  We  asked  them  to 
choose,  if  possible,  a course  they  had  taught  in  1996-97,  were  continuing  to  teach  in  1997-98,  and 
expected  to  teach  in  1998-99. 

The  survey  contained  two  main  sections  concerning  teaching  practices  in  the  selected  course, 
one  on  the  content  taught  and  one  on  pedagogy.  We  discuss  these  two  sections  further  when  we 
present  our  results,  in  subsequent  sections  of  the  chapter. 

Of  the  575  teachers  surveyed  in  the  30  in-depth  case  schools,  436  teachers  (76  percent) 
responded.3  Some  teachers  who  responded  did  not  teach  mathematics  or  science  during  the  1996-97 
school  year,  either  because  they  were  not  employed  as  teachers  in  1996-97  or  because  they  taught 
other  subjects,  and  thus  they  are  not  included  in  the  analyses  of  classroom  teaching.  In  addition,  we 
excluded  some  teachers  from  particular  analyses  because  they  did  not  complete  all  of  the  required 
items.  The  sample  is  74  percent  female  and  18  percent  minority.  Six  percent  of  the  teachers  in  the 
sample  are  novice  teachers,  or  teachers  who  have  taught  the  surveyed  subject  for  two  or  fewer  years. 
(See  Appendix  C for  a more  complete  description  of  the  sample  and  response  rates.) 

Several  features  of  the  sample  should  be  considered  in  interpreting  our  results.  First,  by 
design,  the  sample  of  30  schools  is  disproportionately  high  poverty  (50  percent  of  the  sample  schools 
are  high  poverty,  compared  to  the  national  average  of  25  percent).  This  feature  of  the  sample  is 
useful  in  an  evaluation  of  the  Eisenhower  program,  because  the  program  targets  teachers  in  high- 
poverty  schools.  Throughout  the  analyses,  we  tested  whether  differences  between  teachers  in  highl- 
and low-poverty  schools  are  statistically  significant  (at  the  .05  level);  we  note  these  findings  only 
when  they  are  significant.  Second,  we  selected  the  districts  and  schools  in  the  sample  because  they 
had  adopted  diverse  approaches  to  professional  development,  in  addition  to  traditional  workshops 


1 The  other  source  of  teacher-level  data  collected  as  part  of  this  evaluation  is  the  teacher  activity  survey  discussed  in 
Chapter  3.  In  the  activity  survey,  teachers  are  asked  to  describe  how  professional  development  has  changed  their 
instruction.  The  Longitudinal  Study  of  Teacher  Change,  on  the  other  hand,  will  analyze  changes  in  instruction  over 
time  to  determine  the  impact  of  professional  development  on  instruction.  Because  Longitudinal  Study  teachers  are 
not  judging  the  impact  of  professional  development  themselves,  the  study  minimizes  self-report  bias. 

2 The  remaining  two  waves  ask  teachers  to  describe  their  teaching  in  the  1997-98  and  1998-99  years. 

The  response  rate  of  high  school  teachers  was  higher  than  those  of  elementary  and  middle  school  teachers,  in  part 
because  principals  and  department  chairs  in  high  school  were  more  involved  in  administering  the  survey. 
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and  conferences.  If  such  professional  development  is  more  effective  than  traditional  approaches, 
then  the  teachers’  instruction  in  the  sample  schools  might  be  better  than  that  of  the  average  teacher. 
Finally,  the  Longitudinal  Study  of  Teacher  Change  focuses  on  mathematics  and  science  teachers 
because  they  are  the  primary  participants  in  Eisenhower-assisted  activities.  For  all  of  these  reasons, 
the  sample  is  not  nationally  representative,  but  neither  is  it  extremely  unusual. 

Classroom  Observations 

As  part  of  our  site  visits  to  the  30  in-depth  case  study  schools,  we  conducted  one-time 
classroom  observations  of  two  teachers  in  each  school — usually  one  mathematics  teacher  and  one 
science  teacher.  In  conjunction  with  the  observations,  we  conducted  a brief  pre-observation 
interview  and  a somewhat  longer  post-observation  interview  with  each  of  the  60  teachers  we 
observed.  The  teachers  we  observed  were  selected  by  the  principals  of  the  schools  we  visited,  in  part 
based  on  their  availability  at  the  time  of  our  visit  and  willingness  to  be  observed;  participation  in 
professional  development  was  not  a factor  in  selecting  teachers  to  observe.  Thus,  the  teachers  we 
observed  are  not  necessarily  representative  of  all  teachers  in  the  study  schools. 

We  conducted  the  observations  using  a structured  protocol,  designed  in  part  to  parallel  our 
teacher  survey  instrument.  Prior  to  conducting  the  observations,  we  conducted  a training  session  in 
which  our  site  visitors  observed  videotaped  lessons  and  coded  them  using  our  observation  protocol. 
This  allowed  the  site  visitors  to  develop  a common  understanding  of  the  protocol  and  to  check  inter- 
rater reliability. 

National  Assessment  of  Educational  Progress  (NAEP) 

To  report  on  the  consistency  of  the  content  taught  with  high  standards,  we  needed  to  identify 
an  appropriate  measure  of  high  standards.  The  National  Council  of  Teachers  of  Mathematics 
(NCTM)  and  National  Research  Council  (NRC)  standards  set  a framework  for  important 
mathematics  and  science  concepts  that  should  be  taught  in  the  classroom.  However,  these  standards 
are  at  a level  of  generality  that  makes  quantitive  content  analysis  difficult;  therefore,  we  look  to  the 
National  Assessment  of  Educational  Progress  (NAEP)  to  make  explicit  the  content  focus  of  the 
standards.  The  NAEP  provides  items  that  reflect  this  framework  and  permit  content  analyses  items 
to  determine  relative  emphases  for  mathematics  and  science  content.  In  order  to  develop  a test  that 
would  be  perceived  as  national,  the  National  Center  for  Education  Statistics  has  modeled  the  NAEP 
on  the  professional  associations'  standards  (Reese  et  al.,  1997).  For  example,  30  percent  of  the 
science  assessment  involves  hands-on  performance  exercises  and  50  percent  involves  open-ended 
questions  (NAGB,  1997);  these  also  are  areas  of  emphasis  for  the  standards.  The  high  standards  set 
by  the  test  are  evident  in  the  scores  reported  for  the  1996  science  assessment;  only  three  percent  of 
students  tested  at  the  advanced  level  and  21  to  29  percent  tested  at  or  above  the  proficient  level 
(Raizen,  1998).  As  "the  nation's  report  card,"  the  NAEP  represents  an  appropriate  standard,  although 
admittedly  not  the  only  possible  standard.  Because  the  NAEP  focuses  on  content  and  performance 
goals  consistent  with  standards  developed  by  national  professional  associations,  and  because  the 
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NAEP  establishes  high  expectations  for  achievement,  it  is  reasonable  to  use  the  items  on  the  NAEP 
tests  as  a measure  of  high  instructional  standards.4,5 

Organization  of  Chapter 

The  rest  of  the  chapter  is  organized  in  four  sections.  The  first  section  reviews  current 
standards  and  literature  on  effective  content  and  pedagogy.  The  second  section  describes  the  content 
of  instruction  in  the  30  in-depth  schools,  drawing  on  data  from  the  baseline  wave  of  the  longitudinal 
survey  and  the  classroom  observations.  The  third  section  focuses  on  pedagogy  in  the  30  schools. 
Finally,  in  a brief  concluding  section,  we  draw  together  the  implications  of  our  analysis  of  teaching 
in  the  30  schools. 


Effective  Content  and  Pedagogy 


An  understanding  of  good  mathematics  and  science  instruction  begins  with  a vision  for  the 
classroom.  This  is  a difficult  vision  to  capture  for  two  reasons.  First,  effective  learning  experiences 
differ;  there  is  no  single  model  of  an  ideal  class.  Second,  educators  and  researchers  do  not  know  all 
there  is  to  know  about  ideal  instructional  strategies.  However,  research  has  identified  some  common 
elements  of  “good  instruction”  in  mathematics  and  science.  In  particular,  certain  elements  of  content 
and  pedagogy  improve  student  learning. 

Overall,  effective  instruction  can  be  characterized  by  content  that  is  aligned  with  high 
standards  and  pedagogy  focused  on  active  learning.  Content  includes  both  the  topics  of  instruction, 
such  as  fractions,  and  the  teacher’s  expectations  for  student  performance,  such  as  memorizing  or 
understanding  concepts.  Pedagogy  refers  to  the  types  of  activities  used  in  instruction  and  typically 
includes  dimensions  such  as  whole  class  versus  individual  instruction  or  project  versus  text-based 
instruction. 

Content 

Content  coverage  matters  for  student  learning.  Student  achievement  improves  when  the 
content  of  instruction  is  consistent  with  national  standards  and  assessments  (Cohen  & Hill,  1998; 
Gamoran  et  al.,  1997).  National  standards  for  mathematics  and  science  specify  critical  content  areas 
that  effective  instruction  should  address:  covering  core  topics,  such  as  life  science,  and  developing 
students’  topic  understanding  in  sophisticated  ways,  such  as  making  connections  to  real-world 
situations. 


4 However,  some  performance  goals  for  students — such  as  carrying  out  sustained  work — cannot  be  adequately 
measured  by  a timed,  paper-and-pencil  test  such  as  the  NAEP. 

5 Mathematics  and  science  generally  are  tested  in  every  other  NAEP  administration,  or  every  four  years.  The  data 
used  for  these  analyses  were  the  1996  mathematics  and  science  NAEP  tests.  See  Appendix  D for  a description  of 
the  NAEP  content  analysis. 
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The  National  Council  of  Teachers  of  Mathematics  (NCTM)  developed  standards  for 
mathematics  curricula  (NCTM,  1989)  and  instruction  (NCTM,  1991).6  The  key  content  areas  differ 
by  school  level  (i.e.,  K-4,  5-8,  9-12),  but  generally  focus  on  the  following: 

♦ Numbers  and  operations:  understanding  and  representing  numbers  and  relationships, 
understanding  operations,  and  using  computational  tools  and  strategies. 

♦ Patterns  and  functions:  understanding  types  of  patterns  and  relationships,  using  symbolic 
forms,  and  using  models. 

♦ Algebra:  understanding  basic  concepts  (e.g.,  variable,  expression);  representing  situations 
with  tables,  graphs,  rules,  and  equations;  analyzing  tables  and  graphs;  solving  linear 
equations;  investigating  inequalities;  and  applying  algebraic  methods  to  solve  real-world 
problems. 

♦ Geometry  and  spatial  sense:  analyzing  two-  and  three-dimensional  objects,  using 
different  representational  systems,  recognizing  the  usefulness  of  transformations,  and 
using  visualization  and  spatial  reasoning. 

♦ Measurement:  understanding  attributes,  units,  and  systems  of  measurement,  and  applying 
a variety  of  techniques,  tools,  and  formulas. 

♦ Statistics  and  probability:  posing  questions  and  using  data  to  answer  them,  interpreting 
data,  developing  and  evaluating  inferences,  and  understanding  and  using  notions  of 
chance. 

The  mathematics  standards  also  identify  standards  for  student  performance  that  apply  across 
all  grades: 

♦ Problem  solving:  building  new  knowledge  through  work  with  problems,  developing  a 
tendency  to  use  problem-solving  skills  within  and  outside  mathematics,  using  and 
adapting  varied  strategies  to  solve  problems,  and  reflecting  on  mathematical  thinking. 

♦ Reasoning:  recognizing  reasoning  and  proof  as  important,  making  and  investigating 
mathematical  conjectures,  developing  and  evaluating  mathematical  arguments,  and  using 
various  types  of  reasoning. 

♦ Making  connections:  connecting  different  mathematical  ideas,  understanding  how  ideas 
build  to  form  a coherent  whole,  and  using  mathematics  in  non-mathematical  contexts. 

♦ Communicating:  organizing  mathematical  thinking  to  communicate  with  others, 
expressing  mathematical  ideas  coherently,  considering  the  thinking  of  others,  and  using 
the  language  of  mathematics. 


6 A draft  of  the  revised  1989  standards  was  released  in  1998.  Major  changes  include:  1)  reorganizing  the  grade-level 
breakdown  from  K-4,  5-8,  and  9-12  to  preK-2,  3-5,  6-8,  and  9-12;  2)  relating  process  standards — expectations  for 
student  performance — more  closely  to  content  standards;  3)  adding  the  process  standards  of  representation;  and  4) 
emphasizing  the  development  of  content  strands  (e.g.,  algebra)  over  the  grade  levels  (Romberg,  1998). 
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Reform  in  science  education  has  emphasized  real-world  problems,  investigations  of  natural 
phenomena,  and  linkages  to  other  subjects  rather  than  abstract  knowledge  (Raizen,  1998).  In  setting 
content  standards  for  science,  the  National  Research  Council  (NRC)  identified  certain  content  areas 
as  central  to  teaching  and  learning  science:7 

♦ Physical,  life,  earth  and  space  science:  knowing,  understanding,  and  using  knowledge  of 
matter,  motion  and  forces,  energy,  atoms,  chemical  reactions,  organisms,  cells,  evolution, 
behavior,  earth  systems,  and  the  universe. 

♦ Science  and  technology:  developing  abilities  of  technological  design  and  understanding 
about  science  and  technology. 

♦ Science  in  personal  and  social  perspectives:  understanding  and  making  decisions  on 
personal  and  community  health,  populations,  resources,  the  environment,  and  science  in 
society. 

♦ History  and  nature  of  science:  understanding  the  nature  of  science  from  a historic 
perspective. 

In  addition,  NRC  identified  some  concepts  and  student  performance  standards  that  cross 
content  areas,  such  as  systems,  order,  and  organization;  evolution  and  equilibrium;  and  understanding 
of  and  ability  to  conduct  scientific  inquiry. 

In  setting  standards  for  student  performance,  the  NRC  emphasized  developing  skills  to  do 
scientific  inquiry,  such  as  asking  questions,  collecting  data,  and  developing  explanations.  An 
underlying  premise  of  these  standards  is  to  focus  less  on  "student  acquisition  of  information"  and 
more  on  "student  understanding  and  use  of  scientific  knowledge,  ideas,  and  inquiry  processes" 

(NRC,  1996:  p.  52).  Thus,  the  performance  goal  of  memorizing  material  is  less  central  than  the  goals 
of  understanding  concepts  or  making  connections. 

As  the  standards  imply,  the  organization  of  the  curriculum  within  the  school  also  affects 
students’  learning  experiences.  Past  research  has  suggested  that  there  is  too  much  redundancy  in 
content  from  one  grade  level  to  the  next,  at  least  for  kindergarten  through  eighth  grade.  Compared  to 
other  countries,  the  curriculum  in  the  U.S.  covers  more  topics;  each  year  the  curriculum  expands  to 
incorporate  new  topics  but,  unlike  the  practice  in  other  countries,  topics  are  not  phased  out  of  the 
curriculum  in  successive  grades  (Schmidt,  McNight,  and  Raizen,  1997).  Effective  instruction  entails 
organizing  the  curriculum  so  that  learning  at  each  grade  builds  on  prior  learning,  developing  deeper 
and  more  complex  understandings. 

Pedagogy 

Pedagogy — or  the  way  content  is  presented — also  matters  for  student  learning.  National 
mathematics  and  science  standards  emphasize  the  pedagogical  approach  of  active  instruction.  For 
example,  the  science  standards  advocate  inquiry-based  learning,  in  which  the  teacher  facilitates 


7 The  National  Science  Teachers  Association  (NSTA),  the  American  Association  for  the  Ad vancemenf  of  Science 
(AAAS),  and  NRC  each  developed  standards  documents  (see  AAAS,  1993;  AAAS,  1989;  NSTA,  1992),  The  three 
organizations  agreed  that  NRC  would  be  responsible  for  developing  broad  standards  for  science,  so  the  NRC 
standards  are  the  primary  focus  here  (Raizen,  1998). 
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rather  than  informs  (NRC,  1996).  The  mathematics  standards  stress  instruction  that  builds  on 
students’  experience,  in  which  students  are  actively  engaged  in  wrestling  with  complex  problems 
(NCTM,  1998). 

The  standards  are  based  on  research  that  indicates  that  active  learning  is  especially  effective. 
Students  learn  science  best  when  they  are  active  participants,  engaged  in  activities,  rather  than 
passive  recipients  of  lecture-style  instruction  (Raizen,  1998).  Active  learning  calls  for  students  to  be 
involved  in  creating  their  own  learning  experiences.  Pedagogical  approaches  that  support  active 
mathematics  and  science  learning  include  using  inquiry-based  instruction,  in  which  the  teacher 
facilitates  rather  than  informs,  actively  engaging  students  in  complex  problems  for  which  there  are  no 
simple  solutions,  and  incorporating  multiple  disciplines  in  activities  (NCTM,  1998;  NRC,  1996; 
Raizen,  1998). 

National  standards  in  mathematics  and  science,  consistent  with  research  on  effective 
instruction,  indicate  that  both  content — especially  core  topics  and  complex  performance  goals — and 
pedagogy — especially  active  learning — are  important  to  student  learning.  Clearly,  content  and 
pedagogy  are  interrelated.  While  active  learning  is  especially  student-driven,  it  is  still  coordinated 
around  content — effective  teachers  set  instructional  goals  and  monitor  activities,  intervening  when 
appropriate.  Thus,  while  we  examine  content  and  pedagogy  in  turn  in  this  chapter,  the  two  together 
contribute  to  effective  instruction. 


Section  Findings 

♦ Teachers  tend  to  give  more  emphasis  to  low-level  topics  (e.g.,  number  sense, 
calculation ) than  do  items  on  the  National  Assessment  of  Educational  Progress 
( NAEP ) and  less  emphasis  to  topics  such  as  geometry;  further,  there  is  much 
redundancy  in  topics  covered  from  grade  to  grade. 

♦ Teachers  report  balancing  their  emphases  across  all  six  goals  for  student  performance, 
while  the  NAEP  focuses  more  on  some  of  the  concrete  performance  goals.  Although  this 
suggests  that  teachers  are  emphasizing  more  abstract  performance  goals,  such  as 
understanding  concepts  rather  than  memorizing,  our  observations  suggest  that  these 
performance  goals  are  not  as  deep  as  teachers  report  them  to  be. 

♦ Teachers  cover  content  areas  in  greater  breadth  and  less  depth  than  the  content  assessed 
by  the  NAEP,  especially  in  high-poverty  schools. 

♦ Mathematics  instruction  is  more  highly  aligned  with  items  on  the  NAEP  than  is  science 
instruction,  and  elementary  and  middle  school  instruction  are  more  highly  aligned  than 
is  high  school  instruction. 

In  this  section,  we  examine  whether  the  content  covered  by  teachers  in  our  sample 
emphasizes  high  standards.  We  begin  our  discussion  by  describing  our  data  on  the  content  taught 


Content  Coverage  and  High  Standards 
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and  the  approach  we  have  taken  to  determine  the  degree  to  which  the  content  reflects  high  standards. 
Then,  we  turn  to  our  results. 

Content  and  Alignment  with  Standards 

To  assess  the  consistency  of  the  content  taught  with  national  standards,  we  have  collected 
unusually  fine-grained  information  on  the  content  covered  by  our  sample  of  teachers,  and  we  have 
developed  a unique  strategy  of  measuring  alignment,  drawing  on  the  full  set  of  items  administered  as 
part  of  the  National  Assessment  of  Educational  Progress.  In  the  following  sections,  we  provide  a 
brief  overview  of  these  methods.  (See  Appendix  C for  more  detail.) 

Measuring  the  Content  Taught 

Our  main  data  on  content  come  from  the  baseline  wave  of  the  longitudinal  survey  of  teacher 
change.  We  characterize  the  content  taught  in  terms  of  two  major  dimensions:  the  topics  covered  and 
the  performance  goals  teachers  hold  for  students. 

In  the  content  section  of  the  survey,  we  asked  teachers  to  describe  the  content  they  taught  in 
the  class  they  chose  to  describe,  using  a two-dimensional  matrix.  (Different  forms  of  the  matrix  were 
used  for  elementary,  middle,  and  high  school  mathematics  and  science.  See  Exhibit  2.1  for  a sample 
section  from  the  elementary  mathematics  form  of  the  survey.)  The  matrix  was  initially  developed  by 
Porter  et  al.  (1993)  in  a comprehensive  study  of  mathematics  reform  and  was  revised  for  purposes  of 
the  Eisenhower  evaluation.  Since  then,  the  matrix  has  been  used  in  several  other  studies,  including 
Gamoran  et  al.  (1997). 8 

The  rows  of  the  matrix  contain  a comprehensive  list  of  the  topics  and  subtopics  teachers 
might  cover.  Algebra,  for  example,  is  a topic  in  mathematics,  and  multi-step  equations  is  a subtopic 
under  algebra.  Astronomy  is  a topic  within  science,  and  the  Earth’s  moon  is  a subtopic  under 
astronomy.  Each  subject  area  (i.e.,  mathematics  and  science)  and  each  school  level  (i.e.,  elementary, 
middle,  and  high  school)  has  a unique  set  of  topics  and  subtopics.  The  matrix  for  middle  school 
mathematics,  for  example,  has  nine  topics  and  84  subtopics,  while  the  matrix  for  high  school  science 
has  28  topics  and  191  subtopics. 

The  columns  of  the  matrix  contain  performance  goals  for  students.  Performance  goals 
are  teachers’  expectations  for  what  students  should  be  able  to  do.  There  are  six  performance 
goals  in  the  matrix:  1)  memorize;  2)  understand  concepts;  3)  perform  procedures;  4)  generate 
hypotheses;  5)  collect,  analyze,  and  interpret  data;  and  6)  make  connections.  (See  Exhibit  2.2  for 
definitions  of  the  performance  goals.)  For  example,  when  a teacher  emphasizes  memorizing,  the 
teacher  may  expect  students  to  be  able  to  produce  definitions  or  terms,  facts,  and  formulas  from 
memory.  When  a teacher  emphasizes  using  information  to  make  connections,  the  teacher  may 
expect  students  to  be  able  to  use  and  integrate  concepts,  apply  ideas  to  real-world  situations, 
build  or  revise  theory,  and  make  generalizations. 


8 Porter  et  al.  (1993)  present  comprehensive  information  on  the  reliability  and  validity  of  data  collected  using  the 
content  matrix,  as  well  as  using  teacher  logs  and  classroom  observations. 


2-8 


Excerpt  from  Content  Coverage  Section  of  the  Elementary  School  Mathematics  Teachers  Survey 
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EXHIBIT  2.2 


Performance  Goals  for  Students 


The  following  are  various  performance  goals  that  teachers  hold  for  their  students.  Refer  to  this  list  in  describing 
your  performance  goals  for  each  topic  covered  as  part  of  instruction. 


Memorize 

Deft  n ition  s/T  erms 

Facts 

Formulas 

Understand  Concepts 

Explain  concepts 

Explain  procedures/methods  of  science  & inquiry 
Develop  schema,  or  frameworks  of  understanding 
Represent  concepts  with  manipulatives  or  models 

Perform  Procedures 

Use  numbers 

Do  computation,  execute  procedures  or  algorithms 
Replicate  (illustrative  or  verification)  experiments 
Follow  procedures/instructions 


Collect,  Analyze  & Interpret  Data 

Make  observations 
Take  measurements 
Classify/order/compare  data 
Analyze  data,  recognize  patterns 
Infer  from  data,  predict 
Explain  findings,  results 
Organize  & display  data  in  tables, 
graphs  or  charts 

Use  Information  to  Make  Connections 

Use  & integrate  concepts 
Apply  to  real-world  situations 
Build/revise  theory 
Make  generalizations 


Generate  Questions/Hypotheses 
Brainstorm 
Design  experiments 
Solve  novel/non-routine  problems 


Source : Longitudinal  Teacher  Survey,  Instruction  booklet  for  section  11. 

How  to  read  this  exhibit:  This  exhibit  illustrates  the  instructions  given  survey  respondents  for  completing  the  matrix  displayed  in 
Exhibit  2.1. 


A content  area  can  be  defined  as  the  intersection  of  the  two  dimensions,  topics  and 
performance  goals.  For  example,  if  teachers  emphasize  memorizing  facts  about  the  Earth’s  moon, 
the  content  area  incorporates  the  subtopic  (the  Earth’s  moon)  and  the  performance  goal 
(memorizing).  Both  elements — topics  and  performance  goals — are  integral  to  understanding  the 
content  of  a lesson  or  course.  For  example,  the  student  learning  that  would  be  likely  to  take  place  if 
the  content  were  memorizing  facts  about  the  Earth’s  moon  (e.g.,  gravity,  distance  from  the  Earth)  is 
very  different  from  the  student  learning  that  would  occur  if  the  content  were  understanding  the 
Earth’s  moon  (e.g.,  forces  working  to  keep  satellites  in  orbit). 

Each  teacher  was  asked  to  follow  several  steps  in  describing  the  teacher’s  year-long  course 
using  the  matrix.  First,  the  teacher  indicated  the  amount  of  time  given  to  each  subtopic,  using  a scale 
from  0=no  time  through  3=more  than  two  lessons  or  class  periods.  Then,  the  teacher  indicated  the 
relative  amount  of  emphasis  given  to  each  performance  goal  when  teaching  the  subtopic,  using  a 
scale  from  0=no  emphasis  to  3=sustained  emphasis.  We  used  the  full  matrix  of  data  provided  by 
each  teacher  to  calculate  the  percentage  of  the  teacher’s  total  year-long  class  time  devoted  to  each 
topic  and  subtopic,  each  performance  goal,  and  each  content  area  (intersection  of  a subtopic  and 
performance  goal). 
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Assessing  the  Consistency  with  High  Standards 

We  compare  teachers’  instruction  to  the  NAEP  items  in  order  to  assess  how  well  instruction 
meets  high  standards.  To  determine  the  relative  amount  of  emphasis  given  by  the  NAEP  to  each 
subtopic,  performance  goal,  and  content  area  in  our  elementary,  middle,  and  high  school  science  and 
mathematics  matrices,  we  reviewed  the  full  set  of  NAEP  mathematics  and  science  items  for  the  1996 
tests  for  fourth,  eighth,  and  twelfth  grade.9  We  asked  two  curriculum  experts  in  mathematics  and  two 
experts  in  science  to  review  each  NAEP  item  and  to  determine  the  specific  subtopics  and 
performance  goals  each  item  was  designed  to  tap.10  Using  this  information  for  the  full  set  of  NAEP 
items,  we  computed  the  relative  emphasis  given  by  NAEP  to  each  subtopic,  performance  goal,  and 
content  area. 

In  the  paragraphs  that  follow,  we  draw  on  our  data  provided  by  teachers,  as  well  as  our 
information  from  the  NAEP,  to  examine  four  aspects  of  the  content  taught.  First,  we  examine  the 
extent  to  which  the  topics  covered  by  the  teachers  in  our  sample  match  the  topics  assessed  by  the 
NAEP.  Then,  we  consider  the  extent  to  which  the  performance  goals  our  teachers  emphasize  match 
the  NAEP.  Third,  we  examine  the  content  areas  (intersection  of  topics  and  performance  goals). 
Finally,  we  develop  an  overall  index  of  the  alignment  between  the  content  covered  by  teachers  and 
the  content  assessed  by  the  NAEP. 

Topic  Emphases  and  High  Standards 

In  this  section,  we  examine  the  emphasis  given  by  teachers  in  our  sample  to  specific  topics, 
and  we  compare  this  with  the  relative  emphasis  given  to  the  same  topics  in  the  NAEP.  Research 
indicates  that  some  topics,  for  example  geometry  and  measurement  in  mathematics,  are  special 
weaknesses  for  students  in  the  United  States  (Beaton  et  al.,  1996).  In  the  Third  International 
Mathematics  and  Science  Study,  U.S.  students  in  seventh  grade  scored  19th  out  of  27  countries  in 
geometry  and  23rd  in  measurement.  U.S.  students  in  eighth  grade  scored  21st  out  of  25  countries  in 
geometry  and  23rd  in  measurement  (Beaton  et  al.,  1996).  Thus,  in  interpreting  our  results,  we  give 
special  attention  to  these  two  topics. 

Exhibit  2.3  presents  our  data  on  the  emphasis  given  to  particular  topic  areas,  for  the  middle 
school  mathematics  teachers  in  our  sample.  (Similar  results  for  the  other  teachers  in  our  sample  are 
included  in  Appendix  D.)  The  results  indicate  that  topics  that  traditionally  have  been  weaknesses  for 
U.S.  students,  especially  for  mathematics,  do  not  receive  much  attention  from  teachers  in  our  sample. 
Despite  evidence  that  middle  school  students  need  to  focus  on  measurement  and  geometry,  middle 
school  teachers  surveyed  for  this  study  taught  measurement  on  average  for  12  percent  of  their  course, 
compared  to  the  NAEP’s  emphasis  of  20  percent.  The  average  middle  school  teacher  taught 
geometry  for  nine  percent  of  their  course,  compared  to  the  NAEP’s  emphasis  of  15  percent.  On  the 
other  hand,  middle  school  teachers  tend  to  give  more  emphasis  to  low-level  topics  than  does  the 
NAEP.  For  example,  middle  school  teachers  emphasized  number  sense  37  percent  of  the  time 
compared  to  14  percent  for  the  eighth-grade  NAEP.1 1 


9 The  NAEP  fourth-grade  test  was  compared  to  instruction  for  teachers  in  elementary  school  (grades  K-5).  The 
NAEP  eighth-grade  test  was  compared  to  instruction  for  teachers  in  middle  school  (grades  6-8).  The  NAEP  twelfth- 
grade  test  was  compared  to  instruction  for  teachers  in  high  schools  (grades  9- 1 2). 

0 Appendix  D provides  information  on  the  reliability  of  the  expert  ratings  of  the  NAEP  items. 

11  Results  shown  in  Appendix  D indicate  that  the  emphasis  on  geometry  is  lower  in  high-poverty  schools  than  in 
low-poverty  schools.  - 1. 
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EXHIBIT  2.3 


Percentage  of  Emphasis  on  Topics  in  Middle  School  Math,  Reported  by  NAEP  and  by 
Teachers  in  the  Longitudinal  Teacher  Survey  (n=38) 


Number  Computation  Measurement  Analysis  & Pre-Algebra  Basic  Algebra  Adv  Algebra  Geometry  Technology 
Sense  Prob 


Middle  School  Math  Topics 


Source4.  Longitudinal  Teacher  Survey,  Fall  1997  ( 1996-97  school  year). 

How  to  read  this  exhibit:  The  first  bar  shows  that  NAEF  has  a relative  emphasis  of  14  percent  on  the  topic  Number  Sense  for  middle  school 
math,  and  the  second  bar  shows  that  middle  school  math  teachers  in  our  sample  indicate  a relative  emphasis  of  37  percent  on  the  topic  Number 
Sense.  Each  of  the  topic  areas  for  middle  school  math  is  included  in  the  bar  chart.  The  relative  emphasis  can  be  between  0 and  100  percent. 


Questions  about  the  topic  focus  for  instruction  can  be  extended  from  whether  teachers  cover 
critical  topics  to  whether  they  cover  any  topics.  Previous  research  has  found  that  teachers  sometimes 
focus  so  much  on  changing  the  process  of  instruction  that  they  neglect  the  topics  of  the  lesson. 
Roitman  (1998),  for  example,  described  a case  in  which  an  observed  teacher  was  so  focused  on 
active  learning  activities  that  her  lesson  was  topic-free.  To  consider  this  possibility,  using  the 
Eisenhower  data,  we  turn  to  the  classroom  observations.12  Several  of  the  observed  lessons  did  focus 
on  process  to  the  point  of  having  little  or  no  content  emphasis.  For  example,  in  a sixth-grade  science 
lesson  in  Boonetown,  the  class  focused  on  using  the  scientific  method.  Students  conducted  an 
experiment  to  test  the  absorption  of  different  brands  of  paper  towels,  as  part  of  a consumer  unit.  In 
preparation  for  the  experiment,  student  groups  had  written  hypotheses  and  experimental  designs  the 
day  before.  The  teacher  introduced  the  class,  reviewed  relevant  vocabulary,  and  directed  the  students 
to  work  in  groups.  The  groups  designed  and  conducted  their  own  experiments.  At  the  end  of  the 
lesson,  the  teacher  asked  students  to  consider  ways  in  which  the  experiment  could  be  improved. 
However,  students  did  not  discuss  or  present  their  findings.  While  the  lesson  could  have  developed 


12  The  surveys,  which  required  teachers  to  report  content  foci,  cannot  be  used  to  answer  this  question.  If  the  teacher 
did  not  report  on  content,  the  survey  was  considered  incomplete,  and  the  data  were  not  used. 


understanding  of  material  composition  or  properties,  the  focus  throughout  was  on  process  rather  than 
content. 


The  United  States  differs  from  other  countries  in  how  the  content  is  organized  across  grades. 
In  the  U.S.,  topics  are  repeated  in  many  grades,  theoretically  with  increasingly  complex  subtopics. 
Other  countries,  such  as  France  and  Japan,  focus  on  selected  topics  at  each  grade  level  (Matheson  et 
al.,  1996).  For  example,  the  eighth-grade  mathematics  teachers  in  Japan  focus  on  four  topics,  with 
relatively  little  emphasis  (less  than  four  percent  of  instructional  time)  on  other  topics.  In  contrast, 
U.S.  teachers  spread  instruction  over  21  topics  (Wilson  and  Blank,  1999). 

According  to  national  standards  for  science  and  mathematics,  developed  by  the  National 
Research  Council  (NRC)  and  the  National  Council  of  Teachers  of  Mathematics  (NCTM),  it  is 
possible  to  cover  the  same  broad  topics  across  grades,  while  enhancing  the  depth  of  exposure. 
Revisiting  topics  in  successive  grades  should  build  on  understanding,  allowing  instruction  to  focus 
on  more  complex  subtopics  within  broad  topics. 

The  NRC  science  content  standards  cover  the  same  broad  topics  (i.e.,  physical,  life,  earth  and 
space  science,  science  and  technology,  science  in  personal  and  social  perspectives,  and  history  and 
nature  of  science)  at  all  grade  levels.  These  standards  stress  developing  a more  sophisticated 
understanding  of  more  advanced  subtopics  within  each  topic  as  students  move  up  grade  levels,  to 
reflect  developmental  and  learning  abilities  of  students.  With  this  approach,  the  subtopics  become 
more  abstract  and  students  are  expected  to  develop  greater  conceptual  understanding  as  they  progress 
through  the  grades.  Thus,  NRC’s  science  standards  suggest  that,  at  a gross  level,  there  should  be 
substantial  overlap  in  topics  across  grade  levels;  however,  students  at  each  school  level  should  be 
learning  different  subtopics  within  these  broad  topics. 

Similar  to  the  science  standards,  the  NCTM  standards  emphasize  five  major  topics  (i.e., 
number  and  operations;  patterns,  functions,  and  algebra;  geometry  and  spatial  sense;  measurement; 
and  data  analysis,  statistics,  and  probability)  in  every  grade.  Again,  like  the  science  standards,  the 
mathematics  standards  stress  that  specific  subtopics  within  broad  topics  become  more  sophisticated 
as  the  student  progresses  through  the  grades. 

Consistent  with  the  international  research,  our  data  show  that,  on  average,  teachers  across 
grades  generally  teach  the  same  topics.  There  is  little  clear  pattern  of  intensified  coverage  in  broad 
topic  areas  as  grade  levels  increase.  For  example,  teachers  at  all  grade  levels  teach  measurement  in 
mathematics,  some  grades  more  than  others,  but  there  is  no  pattern  of  consistent  increase  or  decrease 
in  focus  on  measurement  as  grade  levels  increase  (see  Exhibit  2.4).  First-grade  mathematics  teachers 
emphasized  measurement  for  12  percent  of  the  time,  second-grade  teachers  for  23  percent,  sixth- 
grade  for  9 percent,  and  twelfth-grade  for  13  percent. 

Furthermore,  the  measurement  subtopics  emphasized  do  not  consistently  increase  in 
complexity  as  the  grade  level  increases  (see  Exhibit  2.5).  Of  the  16  measurement  subtopics,  only 
four  show  the  expected  pattern.  Two  of  the  more  low-level  subtopics  (i.e.,  use  of  instruments  and 
time  and  temperature)  showed,  on  average,  decreased  emphasis  as  grade  level  increased  (from  34 
percent  to  nine  percent  and  from  23  percent  to  one  percent,  respectively).  Two  of  the  more  complex 
topics  (i.e.,  Pythagorean  theory  and  trigonometry)  showed,  on  average,  increased  emphasis  as  the 
grade  level  increased  (from  zero  to  6 percent  and  from  zero  to  48  percent,  respectively).  For  the 
most  part,  however,  there  is  little  evidence  of  increasingly  complex  topics  in  successive  grades. 
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EXHIBIT  2.4 


Percentage  of  Emphasis  Mathematics  Teachers  in  the  Longitudinal  Teacher  Survey 

Give  to  Measurement,  by  Grade  (n=181) 


Grade 


Source : Longitudinal  Teacher  Survey,  Fall  1997  (1996-97  school  year). 

How  to  read  this  exhibit:  The  first  bar  shows  that  kindergarten  teachers  in  the  longitudinal  teacher  survey  report  a 10  percent  emphasis  on 
measurement.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  emphasis  given  to  measurement  for  teachers  in  each  grade. 
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Relative  Emphasis  on  Subtopics  in  Measurement  by  Grade,  as  Reported  by  Teachers  in 

the  Longitudinal  Teacher  Survey  (n=181) 
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Performance  Goals  and  High  Standards 

In  this  section,  we  compare  the  performance  goals  emphasized  by  teachers  in  our  sample 
with  the  performance  goals  emphasized  in  the  NAEP.  Prompted  in  part  by  the  report  A Nation  at 
Risk  (NCEE,  1983),  there  has  been  a movement  toward  teaching  for  understanding  rather  than 
memorizing  (Roitman,  1998).  National  mathematics  and  science  standards  emphasize  teaching  for 
understanding.  Teaching  for  understanding  and  teaching  for  memorizing  are  performance  goals,  or 
expectations  teachers  have  for  what  students  should  be  able  to  do.  Ideally,  teachers  would  have 
performance  goals  for  students  that  are  consistent  with  the  performance  goals  advocated  in  the 
national  standards.  Of  the  six  performance  goals  teachers  could  identify  on  the  Longitudinal  Teacher 
Survey  (i.e.,  memorize,  understand  concepts,  perform  procedures,  generate  hypotheses, 
collect/analyze/interpret  data,  make  connections),  four  are  especially  relevant  for  the  abstract 
thinking  involved  in  developing  complex  understanding:  understanding  concepts,  generating 
hypotheses,  collecting/analyzing/interpreting  data,  and  making  connections.  Of  the  six  performance 
goals,  two  are  especially  relevant  for  developing  concrete  skills  and  knowledge:  memorizing  and 
performing  procedures.  Ideally,  teachers  will  balance  their  emphasis  on  the  six  performance  goals. 

In  this  section,  we  examine  whether  teachers  have  balanced  performance  goals  for  their  students, 
comparing  teachers’  performance  goals  to  the  emphases  on  the  NAEP. 

Exhibit  2.6  presents  our  results  for  elementary  school  mathematics.  (Results  for  other 
teachers  in  our  sample  appear  in  Appendix  D.)  The  results  indicate  that  NAEP  items  tend  to  focus  on 
the  performance  goals  that  involve  less  abstract  thinking,  such  as  memorizing,  which  does  not 
exemplify  the  ideal  pattern  advocated  by  the  national  standards.  Compared  to  the  performance  goal 
emphases  in  the  NAEP,  teachers  in  our  sample  give  more  equal  emphases  to  all  six  goals.  For 
example,  elementary  mathematics  teachers  report  devoting  20  percent  of  their  class  time  to 
performing  procedures,  compared  to  44  percent  for  the  NAEP,  and  they  devote  17  percent  of  their 
class  time  to  making  connections,  compared  to  5 percent  for  the  NAEP. 
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EXHIBIT  2.6 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 

on  Performance  Goals  (n=74) 


Performance  Goals 


Source:  Longitudinal  Teacher  Survey,  Fall  1997  (1996-97  school  year). 

How  to  read  this  exhibit:  The  First  bar  shows  that  the  NAEP  has  a relative  emphasis  of  1 3 percent  on  the  performance  goal  Memorize,  and  the 
second  bar  shows  that  teachers  in  our  sample  indicate  a relative  emphasis  of  16  percent  on  the  performance  goal  Memorize.  Each  performance 
goal  is  included  in  the  bar  chart.  The  relative  emphasis  can  be  between  0 and  100  percent. 


To  examine  differences  across  types  of  teachers  in  the  emphasis  given  to  the  six  performance 
goals,  we  computed  the  mean  percent  emphasis  for  each  goal  by  school  level  (elementary,  middle, 
and  high  school),  subject  (mathematics  and  science),  and  school  poverty  (high  and  low).  Results 
indicate  that  mathematics  teachers  emphasize  the  more  concrete  performance  goal,  performing 
procedures,  significantly  more  than  science  teachers  do,  while  science  teachers  emphasize  the  more 
abstract  performance  goals — collecting,  analyzing,  and  interpreting  data,  making  connections,  and 
generating  hypotheses — significantly  more  than  mathematics  teachers  do.  Teachers  in  high-poverty 
schools  place  significantly  greater  emphasis  on  memorizing  and  significantly  less  emphasis  on 
understanding  concepts,  compared  to  teachers  in  low-poverty  schools.  High  school  teachers  place 
significantly  greater  emphasis  on  performing  procedures  than  either  elementary  or  middle  school 
teachers,  and  significantly  less  emphasis  on  generating  hypotheses  than  middle  school  teachers;  this 
reinforces  the  finding  noted  earlier,  that  instruction  does  not  seem  to  be  more  complex  or  abstract  at 
higher  grade  levels  (see  Exhibit  2.7). 


Mean  Percent  Emphasis  Given  to  Each  Performance  Goal  (Standard  Deviation), 
by  School  Level,  Subject,  and  Poverty  Level  (n=355) 
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The  emphases  teachers  in  this  study  give  to  the  six  performance  goals  found  in  this  study  are 
inconsistent  with  previous  studies  of  teachers’  instructional  emphases.  In  the  early  1980s,  studies  of 
41  elementary  school  teachers  found  that  70  to  75  percent  of  mathematics  instruction  focused  on 
teaching  students  skills,  such  as  addition,  with  little  attention  to  developing  conceptual  understanding 
or  problem  solving  (Porter,  1989).  In  a groundbreaking  study  of  mathematics  and  science  instruction 
in  high  school,  Porter  et  al.  (1993)  found  that  in  1990  and  1991,  teachers  reported  focusing  most  on 
solving  routine  problems  (e.g.,  computation),  in  both  mathematics  and  science.  The  performance  goal 
of  building  and  revising  theory  and  developing  proofs  was  emphasized  very  little.  Observations 
confirmed  that  teachers  did,  in  general,  focus  on  the  more  rote  performance  goals  such  as 
memorizing  facts,  definitions,  and  equations,  performing  procedures,  and  solving  routine  problems. 
The  TIMSS  videotape  classroom  study  also  found  an  emphasis  on  routine  procedures  in  U.S. 
mathematics  classes  (Stigler  et  al.,  undated).  Teachers  in  the  current  study,  however,  report  no 
greater  emphasis  on  rote  skills,  such  as  memorization,  than  on  conceptual  development  skills,  such  as 
making  connections. 

It  could  be  that  the  low  emphasis  on  rote  skills  reported  here  reflects  teachers’  beliefs  about 
their  instruction  (Cohen,  1990).  For  example,  Knapp  et  al.  found  that  many  teachers  who  were  trying 
to  use  new  instructional  practices  to  improve  student  understanding  “got  the  words  but  not  the  tune,” 
or  used  new  learning  activities  without  understanding  or  capitalizing  on  their  potential  (Knapp  et  al., 
1993:  p.  23);  and  in  a study  of  25  mathematics  teachers  professing  familiarity  with  and  use  of 
standards-based  instructional  strategies,  only  four  truly  demonstrated  the  intent  of  the  standards 
(Spillane  & Zeuli,  1999). 

The  Eisenhower  evaluation  observations  found  examples  of  similar  differences  between 
teachers’  perceptions  of  their  instruction  and  observers’  analyses  of  the  same  classes.  An  elementary 
school  in  Richmond  provides  an  example  of  the  discrepancy  between  a teacher’s  description  of  her 
instruction  and  a trained  observer’s  description  of  the  same  class.  For  the  first  half  of  this  lesson,  the 
teacher  led  a whole-class  review  of  operations  (e.g.,  addition,  subtraction,  exponents,  square  roots). 
She  wrote  pieces  of  equations  on  the  chalkboard  and  asked  students  to  fill  in  the  missing  information. 
While  the  lesson  allowed  students  to  “create”  equations,  the  thinking  tasks  were  defined  within  very 
narrow  parameters,  and  the  focus  of  the  lesson  was  on  arriving  at  the  “correct  answer.”  The  teacher 
felt  that  she  was  emphasizing  complex  performance  goals  such  as  interpreting  data  and  making 
connections  for  more  than  half  of  this  part  of  the  lesson  (66  percent).  The  two  observers,  however, 
saw  an  emphasis  on  lower  level  performance  goals,  such  as  performing  procedures  and  memorizing 
(60  percent). 

A middle  school  in  Rhinestone  provides  an  example  of  a lesson  that  emphasizes  more 
complex  performance  goals.  In  this  middle  school  science  lesson,  students  constructed  a bridge  out 
of  spaghetti  and  marshmallows.  The  bridge  had  to  meet  certain  specifications  of  length  and  width 
and  have  certain  characteristics  (e.g.,  it  had  to  have  two  piers).  According  to  the  teacher,  this  project 
was  intended  to  make  more  concrete  the  construction  problems  students  had  been  discussing  in 
previous  classes.  The  class  was  part  of  a larger  unit  on  bridges  and  other  structures,  and  the  unit  was 
part  of  an  inquiry-based  curriculum  designed  by  the  observed  teacher  and  a peer. 

After  the  class,  the  teacher  and  the  observer  separately  identified  the  performance  goals  and 
topics  emphasized  in  the  class.  Both  the  teacher  and  the  observer  felt  the  lesson  stressed  performance 
goals  that  had  to  do  with  developing  complex  understanding.  The  teacher  and  observer  gave 
favorable  descriptions  of  the  class.  Both  agreed  on  the  percentage  of  emphasis  the  lesson  gave  to 
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memorizing.  But  the  teacher  felt  that  she  placed  more  emphasis  on  understanding  content  than  the 
observer  thought  she  did. 

Clearly,  in  some  cases,  teachers  are  emphasizing  performance  goals  of  increased  student 
understanding.  However,  in  some  cases,  changes  in  instruction  may  not  be  at  the  level  that  would 
improve  student  understanding. 

These  observations  raise  the  question  of  the  prevalence  of  teacher  exaggeration  in  their 
reported  instruction.  Do  teachers,  in  general,  tend  to  over-report  performance  goals  that  they  see  as 
positive?  The  second  example  shows  that  teachers  do  not  consistently  exaggerate.  Yet  the  first 
example  shows  that  some  teachers  may  report  a more  favorable  picture  of  their  instruction  than  is 
observed.  Although  we  cannot  conclusively  determine  the  accuracy  of  teachers’  reporting  from  these 
data,  the  self-reporting  bias,  if  any,  should  not  have  much  influence  on  the  analyses  of  change  in 
teaching  practice  to  be  discussed  in  our  third  report,  because  we  would  expect  any  bias  to  be  constant 
across  the  three  waves  of  the  survey. 

As  previous  research  suggests,  there  may  be  a gap  between  the  teacher’s  perception  of  her 
instruction  and  a more  objective  evaluation  of  the  same  lesson.  Teachers  may  believe  they  are 
teaching  in  ways  consistent  with  high  standards;  without  feedback  on  their  instruction,  they  may  not 
recognize  areas  for  improvement.  Well-constructed  professional  development,  which  provides 
opportunities  for  such  feedback,  may  help  teachers  continue  to  evaluate  and  improve  their 
instruction. 

Content  Emphases:  The  Intersection  of  Topics  and  Performance  Goals 

In  this  section,  we  turn  to  the  emphasis  teachers  give  to  specific  content  areas — that  is,  to  the 
intersection  of  topics  and  performance  goals.  As  noted  above,  the  idea  of  “content”  includes  both  the 
topic  of  instruction  and  the  teacher’s  goals  for  student  performance. 

The  mathematics  and  science  standards  present  a vision  for  instruction,  in  which  each  grade 
builds  on  the  learning  in  the  previous  grade.  However,  research  on  teaching  practice  and  findings 
reported  earlier  from  the  current  study  suggest  that  subtopics  and  performance  goals  do  not  become 
more  challenging  as  students  move  through  the  grades.  Rather,  students  visit  and  revisit  the  same 
topics  and  subtopics  at  a superficial  level.  This  curriculum  organization  contributes  to  a phenomenon 
in  the  U.S.  recognized  as  “teaching  for  exposure”  (Porter,  1989).  Because  many  topics  and  subtopics 
are  taught  at  more  than  one  grade  level,  teachers  provide  very  limited  instruction  in  a large  number  of 
topics  and  subtopics.  This  practice  is  unlikely  to  deepen  students’  understanding  of  any  particular 
topic  (Rollefson,  1996).  Effective  instruction  must  balance  covering  a variety  of  content  areas 
(breadth)  with  developing  deep  understanding  in  each  content  area  (depth),  perhaps  even 
emphasizing  depth  by  limiting  breadth  somewhat  (Raizen,  1998). 

To  assess  the  depth  and  breadth  of  content  covered,  we  counted  the  total  number  of  possible 
content  areas  (cells)  in  our  matrix.  We  then  determined  the  percentage  of  cells  that  teachers  reported 
covering,  and  compared  this  percentage  with  content  areas  (cells)  assessed  by  the  NAEP.  Our 
results,  shown  in  Exhibit  2.8,  indicate  that  teachers’  instruction  shows  greater  breadth  than  is 
reported  on  the  NAEP.  While  the  NAEP  does  not  assess  and  the  surveyed  teachers  do  not  cover  all 
possible  content  areas,  teachers  consistently  cover  substantially  more  content  areas  than  represented 
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in  the  NAEP,  in  some  cases  as  much  as  twice  as  many  areas.13  For  example,  teachers  cover  39 
percent  of  the  144  content  areas  in  middle  school  science,  while  the  NAEP  assesses  only  16 
percent.14  Teachers  in  high-poverty  schools  tend  to  cover  more  content  areas  than  teachers  in  low- 
poverty  schools. 


EXHIBIT  2.8 

Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Coverage 

of  Content  Areas  (n=355) 


Mathematics 

i 

Science 

Elementary 

School 

Middle 

School 

High 

School 

Elementary 

School 

Middle 

School 

High  School 

Number  of  possible 
content  areas 

60 

54 

96 

96 

144 

168 

Percent  assessed  by 
NAEP 

35% 

34% 

20% 

26% 

16% 

15% 

Percent  covered  by 
teachers 

47% 

51% 

43% 

36% 

39% 

32% 

Source : Longitudinal  Teacher  Survey,  Fall  1997  (1996-97  school  year). 

How  to  read  this  exhibit:  The  first  column  shows  that  for  elementary  school  mathematics,  there  are  60  possible  content  areas.  Thirty-five 
percent  of  the  possible  content  areas  were  assessed  by  NAEP.  Forty-seven  percent  of  the  content  areas  are  covered  by  teachers  in  our 
sample.  Coverage  of  content  areas  data  are  listed  in  the  rows.  Grade  levels  are  in  the  columns.  The  possible  coverage  can  be  between  zero 
and  100  percent. 


Covering  a large  number  of  content  areas  is  not  necessarily  an  instructional  liability; 
however,  if  by  covering  more  content  areas  teachers  are  unable  to  focus  on  each  content  area  as 
thoroughly  as  needed,  students  may  not  have  opportunities  to  develop  deep  understandings  in  each 
area.  Our  data  suggest  this  is  the  case.  In  comparison  to  the  NAEP  items,  teachers  cover  more 
content  areas,  giving  many  relatively  little  emphasis.  For  example,  Exhibit  2.9a  shows  that  in 
elementary  school  science,  the  NAEP  gives  strong  emphasis  (greater  than  3 percent  emphasis)  to  1 1 
content  areas;  Exhibit  2.9b  shows  that  teachers  report  strong  emphasis  on  only  three  content  areas. 
On  the  other  hand,  the  NAEP  gives  weak  emphasis  (1  to  2 percent  emphasis)  to  only  15  content 
areas,  shown  in  Exhibit  2.9a,  while  teachers  report  a weak  emphasis  on  27  areas.  This  pattern  is 
found  across  subjects  (i.e.,  mathematics,  science)  and  school  levels  (i.e.,  elementary,  middle,  high 
school).  As  previous  research  suggests,  and  the  current  analysis  reiterates,  teachers  tend  to  favor 
breadth  over  depth  in  their  instruction. 


13  Some  performance  goals  that  are  especially  difficult  to  measure  on  a timed,  paper-and-pencil  test,  such  as 
generating  hypotheses,  may  be  underrepresented  on  the  NAEP. 

4 This  difference  might  be  due,  in  part,  to  the  fact  that  the  teachers  reported  on  all  content  areas  covered  over  the 
course,  while  the  NAEP  only  tests  on  a sample  of  content  areas  that  students  are  expected  to  learn.  However,  the 
analysis  included  the  full  set  of  NAEP  items,  which  is  a sizable  sample  of  items,  so  NAEP  content  coverage  should 
be  substantial. 
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EXHIBIT  2.9a 


Emphasis  on  Content  Areas  in  Fourth-Grade  Science  NAEP  Items 


Memorize 


Understand 


Perform 

procedures 


Generate 

hypothesis 


Collect 

data 


Make 
connections 


Nature  of  Science 


Technology 


Meas  & Calc 


Living  Systems 


Plants 


Animals 


Humans 


Growth,  Dev,  Beh 


Ecology 


Energy 


Electricity 


Waves 


Matter 


Earth  Systems 


Astronomy 


Meteorology 


NMEAN 


under  .01 


.01  to  .02 


.02  to  .03 


over  .03 


Source:  NAEP,  1996. 

How  to  read  this  exhibit:  The  rows  represent  topic  areas  and  the  columns  performance  goals.  Each  cell  is  a topic  area  and  performance 
goal  combination.  Blank  boxes  indicate  less  than  one  percent  of  relative  emphasis  for  a particular  topic  and  performance  goal  combination 
by  NAEP.  Boxes  with  horizontal  lines  indicate  between  one  and  two  percent  of  relative  emphasis  for  a topic  and  performance  goal 
combination  by  NAEP.  Checkered  boxes  indicate  two  to  three  percent  of  relative  emphasis  for  a topic  and  performance  goal  combination 
by  NAEP.  Shaded  boxes  indicate  over  three  percent  of  relative  emphasis  for  a topic  and  performance  goal  combination  by  NAEP.  For  the 
Nature  of  Science,  under  one  percent  of  the  relative  emphasis  given  by  NAEP  items  was  on  the  performance  goal  Memorize. 


ERjt 


2-22 


93 


EXHIBIT  2.9b 


Emphasis  on  Science  Content  Areas,  Reported  by  Elementary  School  Teachers 
in  the  Longitudinal  Teacher  Survey  (n=69) 


TMEAN 


under  .01 


.01  to  .02 


.02  to  .03 


over .03 


Source : Longitudinal  Teacher  Survey,  Fall  1997  (1996-97  school  year).  Kindergarten  through  fifth  grade  teachers. 

How  to  read  this  exhibit:  The  rows  represent  topic  areas  and  the  columns  performance  goals.  Each  cell  is  a topic  area  and  performance 
goal  combination.  Blank  boxes  indicate  less  than  one  percent  of  relative  emphasis  for  a particular  topic  and  performance  goal  combination 
by  teachers.  Boxes  with  horizontal  lines  indicate  between  one  and  two  percent  of  relative  emphasis  for  a topic  and  performance  goal 
combination  by  teachers.  Checkered  boxes  indicate  two  to  three  percent  of  relative  emphasis  for  a topic  and  performance  goal  combination 
by  teachers.  Shaded  boxes  indicate  over  three  percent  of  relative  emphasis  for  a topic  and  performance  goal  combination  by  teachers.  For 
the  Nature  of  Science,  under  one  percent  of  the  relative  emphasis  given  by  teachers  in  the  longitudinal  teacher  survey  was  on  the 
performance  goal  Memorize. 
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Alignment  between  Content  Emphases  and  High  Standards 

Finally,  in  this  section,  we  report  an  overall  measure  of  the  alignment  between  the  content 
areas  taught  by  teachers  in  our  sample  and  the  content  areas  emphasized  by  the  NAEP.  We 
computed  the  measure  based  on  the  relative  emphasis  given  to  each  content  area  in  our  matrix  (each 
cell)  by  the  teachers  and  by  the  NAEP.15  For  each  teacher,  the  index  takes  on  a value  ranging  from 
zero  (no  agreement  at  all  between  the  content  areas  the  teacher  emphasizes  and  those  emphasized  by 
the  NAEP)  to  100  percent  alignment  (complete  agreement  between  the  content  areas  emphasized  by 
the  teacher  and  the  NAEP).  High  alignment  indicates  that  teachers  emphasize  topics  and 
performance  goals  that  were  similar  to  NAEP’s  emphasis.  For  example,  teachers  might  focus 
especially  on  understanding  concepts  (a  performance  goal)  for  motion  and  forces  (a  topic)  by  asking 
students  to  explain,  in  everyday  terms,  the  relationship  between  motion  and  force.  If  the  NAEP  also 
emphasizes  understanding  concepts  for  motion  and  forces,  there  would  be  high  agreement  between 
instruction  and  the  NAEP  on  that  content  area.  If  there  were  a pattern  of  such  agreement  across 
content  areas,  then  the  index  of  alignment  would  be  high. 

Depending  on  the  subject  (i.e.,  mathematics  or  science)  and  the  school  level  (i.e.,  elementary, 
middle,  or  high  school),  the  average  alignment  between  the  surveyed  teachers’  instruction  and  the 
NAEP  ranges  from  1 1 to  29  percent  (see  Exhibit  2.10).  Considering  the  large  number  of  content 
areas,  alignment  of  29  percent  is  quite  high.  There  is  no  significant  difference  in  alignment  between 
high-  and  low-poverty  schools. 


EXHIBIT  2.10 


Degree  of  Alignment  between  Teachers’  Instructional  Emphases 
and  NAEP  Emphases  (n=355) 
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Source : Longitudinal  Teacher  Survey,  Fall  1997  ( 1996-97  school  year). 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average  teachers’  instructional  emphases  is  aligned  19  percent  with  the  NAEP 
emphases.  Each  dot  represents  one  teacher  in  our  sample.  As  the  number  of  teachers  at  one  data  point  (or  value)  increases,  the  dots  form  a 
horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular  category.  The  number  to  the  right  of 
the  distribution  is  the  mean. 


15  The  index  of  alignment  is  computed  as  the  sum,  across  content  areas,  of  the  absolute  value  of  the  difference 
between  the  teacher’s  and  the  NAEP  emphasis  in  each  content  area,  divided  by  two,  subtracted  from  one;  the  result 
is  multiplied  by  100.  The  absolute  value  is  required  because  the  index  is  designed  to  capture  cells  for  which  the 
teachers  give  more  emphasis  than  the  NAEP,  as  well  as  those  for  which  they  give  less  emphasis. 
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As  can  be  seen  in  Exhibit  2.10,  the  content  of  instruction  for  teachers  in  our  sample  is  better 
aligned  with  the  NAEP  in  mathematics  than  in  science.  There  are  a number  of  possible  reasons  for 
this.  Mathematics  often  is  seen  as  a core  subject;  everyone  generally  agrees  that  children  should 
learn  fractions  or  geometry.  Science,  however,  is  not  always  so  central  to  the  curriculum,  especially 
in  the  early  grades  (Kennedy,  1998;  Raizen,  1998).  At  all  levels,  students  have  less  exposure  to 
science  than  to  mathematics:  30  minutes  per  day  for  science  compared  to  60  minutes  per  day  for 
mathematics  in  elementary  schools,  and  two  to  three  years  of  mathematics  compared  to  two  years  of 
science  required  in  high  school  (Weiss,  1997).  Further,  there  is  greater  national  consensus  on  core 
topics  in  mathematics  than  in  science:  state  mathematics  curricula  tend  to  focus  on  common  topics, 
while  there  is  little  overlap  between  states  in  terms  of  science  emphasized  by  the  state  curricula 
(Schmidt  et  al.,  1996). 

Finally,  teachers  may  be  less  familiar  with  science,  and  this  could  affect  the  quality  of 
instruction.  Elementary  school  teachers  generally  have  a basic  understanding  of  reading  and 
mathematics,  and  feel  comfortable  teaching  these  subjects;  however,  they  may  be  less  knowledgeable 
about  and  comfortable  teaching  science.  Research  suggests  that  teachers’  content  knowledge  affects 
their  instruction  (Rollefson,  1996).  A survey  of  elementary  school  mathematics  and  science  teachers 
found  that  60  percent  felt  qualified  to  teach  mathematics,  28  percent  felt  qualified  to  teach  life 
sciences,  and  fewer  than  10  percent  felt  qualified  to  teach  physical  science  (Weiss,  1997).  Teachers 
teaching  out-of-field  may  misrepresent  key  concepts  or  focus  on  trivial  rather  than  central  concepts 
and  tend  to  rely  on  drill-and-practice  activities  rather  than  instruction  oriented  toward  student 
inquiry.  Further,  because  the  science  standards  are  relatively  new,  teachers  and  students  have  had 
limited  time  to  become  familiar  with  them. 

Our  data  also  indicate  that  content  was  more  highly  aligned  with  the  NAEP  in  the  elementary 
and  middle  schools  than  in  the  high  schools.  This  phenomenon  might  be  an  artifact  of  the  test. 
Although  the  NAEP  is  used  as  a standard  for  high  expectations,  the  high  school  test  is  geared  toward 
content  covered  prior  to  high  school.  Thus,  the  standard  set  by  the  high  school  NAEP  test  might  not 
be  as  challenging  as  the  instruction  of  teachers  who  participated  in  the  Eisenhower  evaluation. 
Although  most  of  the  high  school  teachers  in  the  sample  described  average-level  courses,  such  as 
algebra  and  biology,  some  did  describe  advanced  courses  such  as  calculus  and  physics,  and  honors 
courses  (see  Appendix  D). 

Our  data  indicate  that  the  degree  of  alignment  of  the  content  taught  with  high  standards 
seems  to  be  related  to  the  school  in  which  instruction  occurs.  As  much  as  30  percent  of  the 
difference  among  surveyed  teachers  in  terms  of  how  well  their  instruction  meets  high  standards  can 
be  attributed  to  the  school  in  which  the  teacher  teaches  (see  Appendix  D).  Our  data  indicate  that  the 
effect  of  the  school  on  alignment  with  the  NAEP  is  greater  for  science  than  for  mathematics 
instruction,  and  greater  for  elementary  and  middle  schools  than  high  schools.  These  findings  imply 
that,  at  least  for  this  sample,  strategies  to  help  teachers  improve  instruction  should  be  targeted  to 
schools. 

Finally,  although  we  talk  about  average  alignment  across  groups  of  teachers,  teachers  differ 
from  each  other  in  how  closely  their  instruction  aligns  with  the  NAEP.  This  variation  is  quite  visible 
in  Exhibit  2.10,  which  shows  that  some  teachers  are  nearly  50  percent  aligned  with  the  NAEP,  while 
others  have  almost  no  alignment.  For  example,  the  instruction  of  one  elementary  school  science 
teacher  (called  Teacher  A)  was  minimally  aligned  with  the  NAEP:  on  an  index  of  zero  to  100,  with 
100  indicating  perfect  alignment,  she  had  alignment  of  less  than  one  percent.  On  the  other  hand,  the 
instruction  of  another  elementary  school  science  teacher  (called  Teacher  B)  was  relatively  highly 
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aligned  with  the  NAEP:  in  the  alignment  index,  she  had  alignment  of  35  percent.  The  performance 
goals  and  topics  that  Teacher  B emphasized  resembled  the  NAEP  much  more  closely  than  the 
performance  goals  and  topics  of  Teacher  A.  Teacher  B stressed  memorizing  and  understanding,  as 
the  NAEP  does,  while  Teacher  A stressed  performing  procedures  and  collecting/analyzing/ 
interpreting  data,  unlike  the  NAEP.  Teacher  B emphasized  components  of  living  systems,  ecology, 
properties  of  matter,  and  astronomy  similarly  to  the  NAEP,  while  Teacher  A emphasized 
maintenance  in  animals,  unlike  the  NAEP.  A teacher  such  as  Teacher  B,  whose  instruction  already 
meets  high  standards,  may  not  need  the  same  type  of  guidance  as  a teacher  such  as  Teacher  A,  whose 
instruction  is  weak  compared  to  the  standards. 

Summary:  Content  Coverage  and  High  Standards 

Data  from  the  baseline  Longitudinal  Teacher  Survey  indicate  that  the  content  taught  by 
teachers  in  our  sample  is  moving  toward  but  does  not  yet  meet  high  national  standards  in  several 
ways.  Teachers  do  not  focus  on  some  advanced  topics;  rather,  they  emphasize  low-level  topics. 
Although  teachers  set  more  complex  performance  goals  for  their  students  than  they  have  historically, 
the  changes  in  their  instruction  are  not  always  as  deep  as  the  teachers  perceive  them  to  be.  Teachers 
appear  to  teach  for  exposure,  and  the  content  covered  does  not  appear  to  become  substantially  more 
challenging  in  successive  grades.  These  findings  are  generally  consistent  with  previous  research, 
which  suggests  that  instruction  in  American  schools  does  not  emphasize  challenging  content. 

Targeted  professional  development  can  help  address  these  concerns.  For  example, 
professional  development  that  focuses  on  content  can  help  teachers  develop  a deeper  understanding 
of  the  content  they  teach  and  develop  lessons  that  are  rich  in  challenging  content  (Kennedy,  1998). 
Professional  development  with  in-classroom  follow-up  components  could  help  teachers  understand 
the  level  at  which  their  instruction  has  changed  and  the  areas  in  which  the  intended  change  is  still 
superficial.  (See,  for  example,  Schifter,  1996).  School-based  professional  development  that  includes 
the  collective  participation  of  groups  of  teachers  from  the  same  grade  could  help  teachers  organize 
instructional  emphases  across  grades,  so  that  each  successive  grade  builds  on  the  previous  one.  Later 
chapters  examine  the  prevalence  of  these  types  of  designs  and  characteristics  in  Eisenhower-assisted 
professional  development  activities  and  their  relationship  to  teacher  outcomes. 


Pedagogy  and  High  Standards 


Section  Findings 

♦ Science  teachers,  elementary  school  teachers,  and  teachers  in  low-poverty  schools  report 
using  more  nontraditional  pedagogy  compared  to  mathematics  teachers,  middle  and  high 
school  teachers,  and  teachers  in  high-poverty  schools. 

♦ Teachers  who  emphasize  concrete  performance  goals  for  their  students,  such  as 
memorizing,  tend  to  report  using  traditional,  teacher-centered  pedagogy.  Teachers  who 
give  more  emphasis  to  abstract  performance  goals  for  their  students,  such  as 
understanding  concepts,  tend  to  report  using  active,  project-centered  pedagogy. 
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To  this  point,  we  have  focused  on  the  content  of  instruction:  the  topics  and  performance  goals 
that  teachers  emphasize,  and  how  they  compare  to  high  standards  for  content.  However,  the  delivery 
of  instruction — the  pedagogy — also  is  important  to  effective  teaching.  Students  learn  best  when  they 
are  actively  involved  in  learning,  when  assessment  tools  are  used  to  tailor  the  lesson  to  the  students’ 
individual  needs,  and  when  students  have  access  to  a variety  of  tools  and  modes  for  learning. 
However,  according  to  the  literature,  traditional  instruction  tends  to  be  led  by  the  teacher  and  de- 
emphasizes  student-initiated  activities  other  than  highly  structured  individual  seatwork  (for  example, 
completing  exercises  in  a text  or  on  worksheets)  (Gamoran,  Secada,  & Marrett,  1998).  Traditional 
instruction  entails  at  least  two  dimensions:  an  emphasis  on  teacher  presentation  and  highly  structured 
lessons  using  traditional  materials  (e.g.,  texts).  Conversely,  nontraditional  instruction  revolves 
around  student-initiated  activities,  including  projects  and  discussion,  and  the  use  of  innovative 
materials  (e.g.,  manipulatives).  Research  suggests  that  teachers  are  beginning  to  explore 
nontraditional  pedagogy,  and  to  use  a mix  of  traditional  and  nontraditional  pedagogy  in  their 
instructional  practice  (Spillane  & Zeuli,  1999). 

In  this  section  of  the  chapter,  we  describe  and  analyze  the  pedagogical  approaches  of  teachers 
in  our  sample.  We  conducted  analyses  on  a series  of  six  questions  in  the  Longitudinal  Survey  about 
teachers'  pedagogical  strategies.  Based  on  these  analyses,  we  first  describe  these  teachers'  classes:  the 
average  percent  of  time  teachers  and  students  spend  in  various  types  of  activities,  the  average  relative 
emphasis  on  different  types  of  tasks  and  grouping  of  students.  Next,  we  identify  pedagogical 
patterns  in  the  class  activities,  such  as  didactic  instruction  or  active,  project-centered  instruction.  We 
look  at  patterns  within  our  sample,  for  example  identifying  where  elementary,  middle,  and  high 
school  teachers  differ  in  their  pedagogical  approaches.  (See  Appendix  D for  details  on  analysis 
methods.)  Finally,  we  look  at  the  relationship  between  pedagogy  and  content,  in  order  to  establish 
empirically  whether  certain  pedagogical  approaches  are  more  consistent  with  high  standards  for 
content  than  others. 

Class  Activities 

Class  activities  provide  a picture  of  students'  learning  experiences.  To  some  degree,  the 
emphasis  on  some  types  of  activities  can  show  how  consistently  teachers  are  using  the  types  of 
activities  advocated  by  the  research. 

The  teachers  in  this  study  spent  one-third  of  their  instructional  time  on  teacher-led  activities 
such  as  lecturing  or  providing  demonstrations,  and  two-thirds  of  their  instructional  time  on  activities 
that  could  actively  involve  students,  such  as  whole  class  discussions,  small  group  and  individual 
work,  and  hands-on  experiences.  Exhibit  2.1  la  shows  that  a high  percentage  of  teachers  (23  to  60 
percent)  have  students  do  traditional  activities,  such  as  working  on  homework,  pencil-and-paper 
exercises,  and  taking  quizzes,  in  most  or  all  lessons.  Exhibit  2.1  lb  shows  that  a lower  percentage  of 
teachers  (12  to  31  percent)  have  students  work  on  nontraditional  activities,  such  as  independent  long- 
term projects  or  problems  for  which  there  is  not  an  immediate  solution,  for  most  or  all  lessons. 
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EXHIBIT  2.11a 

Percent  of  Teachers  Using  Traditional  Activities  in  Most  or  All  Lessons  (n=355) 
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Source : Longitudinal  Teacher  Survey,  Fall  1997  ( 1996-97  school  year). 

How  to  read  this  exhibit : The  first  bar  shows  that  percent  of  teachers  report  using  in  most  or  all  lessons.  Each  bar  and  the  number  on  top 
of  it  represent  the  percent  of  teachers  who  report  using  that  activity  in  most  or  all  lessons.  
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EXHIBIT  2.11b 


Percent  of  Teachers  Using  Nontraditional  Activities  in  Most  or  All  Lessons  (n=355) 
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Source : Longitudinal  Teacher  Survey,  Fall  1997  (1996-97  school  year). 

How  to  read  this  exhibit : The  first  bar, shows  that  percent  of  teachers  report  using  in  most  or  all  lessons.  Each  bar  and  the  number  on  top 
of  it  represent  the  percent  of  teachers  who  report  using  that  activity  in  most  or  all  lessons. 


Pedagogical  Patterns 

Research  suggests  that  teachers  tend  to  favor  particular  approaches  to  pedagogy,  with  certain 
types  of  class  activities  occurring  together  (Peterson,  Fennema,  Carpenter,  & Loef,  1989;  Stein, 
Baxter,  & Leinhardt,  1990).  While  professional  development  can  focus  on  changing  individual 
classroom  activities,  such  as  seatwork,  professional  development  is  more  likely  to  have  a lasting 
impact  if  it  targets  the  teacher's  broader  pedagogical  approach  (Hyde,  Ormiston,  & Hyde,  1994). 
Therefore,  in  this  section,  we  identify  general  pedagogical  approaches  that  represent  the  ways 
teachers  organize  classroom  practices. 

We  conducted  a factor  analysis  on  the  survey  questions  noted  above  to  identify  pedagogical 
patterns  in  instructional  activities.  Consistent  with  the  research  on  pedagogy,  we  found  four  distinct 
patterns  of  activities. 


We  developed  a set  of  four  scales  to  measure  the  extent  to  which  teachers  rely  on  these  four 
pedagogical  strategies.  Each  scale  is  scored  to  have  a mean  of  50  for  the  teachers  in  our  sample. 

Two  measures  capture  traditional  strategies:16 

1 . Didactic  instruction:  Didactic,  or  teacher-led,  instruction  includes  the  following  activities: 
lecturing  and  having  students  take  a passive  role;  de-emphasizing  group  work,  reading, 
writing,  and  student  presentations;  and  not  using  concrete  models  or  interdisciplinary 
lessons. 

2.  Individual  seatwork:  Individual  seatwork  includes  the  following  activities:  working  on 
homework  in  class,  working  on  pencil/paper  exercises,  reciting  or  drilling  orally,  taking 
quizzes,  and  having  students  work  individually,  or  in  pairs. 

Two  measures  capture  nontraditional  strategies: 

1.  Active,  project-centered  instruction:  This  factor  includes  the  following  activities:  students 
working  on  independent,  long-term  projects,  problems  with  no  immediate  solution,  and 
technical  writing  skills;  using  hands-on  activities;  de-emphasizing  paperwork  and 
individual  work. 

2.  Discussion-oriented  instruction:  This  measure  focuses  on  discussion  only. 

The  pedagogical  approaches  we  identified  here  are  consistent  with  the  research  on  pedagogy, 
giving  us  confidence  that  the  pedagogy  of  the  teachers  in  this  sample  is  fairly  representative.  The 
national  standards,  as  well  as  research  on  pedagogy,  indicate  that  effective  instruction  calls  for  an 
increased  emphasis  on  nontraditional  pedagogical  approaches,  without  fully  abandoning  traditional 
approaches.  The  degree  to  which  individual  teachers  emphasize  those  pedagogical  approaches  is  one 
indicator  of  their  consistency  with  national  standards. 

Pedagogical  Patterns  within  the  Sample 

According  to  the  research,  pedagogy  differs  substantially  by  type  of  school  and  subject.  For 
example,  high  schools  tend  to  have  more  highly  structured  lessons,  with  greater  emphasis  on 
individual  work,  compared  to  elementary  schools.  However,  national  mathematics  and  science 
standards  consistently  emphasize  active  instruction  for  all  students,  regardless  of  school  level,  school 
poverty,  or  subject.  Therefore,  it  is  useful  to  understand  how  subgroups  of  schools  and  students 
differ  in  their  use  of  effective  pedagogy.  To  examine  these  questions,  mean  scores  were  computed 
on  the  four  pedagogy  scales  by  school  level  (elementary,  middle,  and  high  school),  subject 
(mathematics  and  science),  and  poverty  level  (high  and  low). 

We  found  some  differences  among  types  of  teachers  and  types  of  schools  in  the  pedagogy 
favored,  as  shown  in  Exhibit  2.12.  Mathematics  teachers  are  significantly  more  likely  to  use  seatwork 
(traditional)  and  science  teachers  are  significantly  more  likely  to  use  active  instruction  (non- 


16  We  also  explored  a measure  of  technology  use  (e.g.,  computers  and  calculators),  because  previous  research 
suggests  that  teachers  traditionally  use  technology  for  drill  and  practice  on  facts  and  skills,  but  that  they  may  be 
learning  to  use  technology  for  more  advanced  learning  goals.  However,  responses  to  the  technology  questions 
seemed  to  reflect  resources  (e.g.,  greater  or  lesser  access  to  technology)  rather  than  pedagogy  (e.g.,  how  technology 
is  used  in  the  classroom).  Therefore,  it  was  not  appropriate  to  discuss  the  technology  factor  in  this  section. 
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traditional);  this  is  consistent  with  findings  from  the  observed  sites,  where  mathematics  lessons 
sometimes  revolved  around  textbook  work  and  science  lessons  often  involved  labs.  Compared  to 
elementary  school  teachers,  high  school  teachers  use  more  didactic  instruction  and  individual 
seatwork  (traditional),  and  less  discussion-oriented  instruction  (nontradidonal).  Low-poverty  schools 
are  significantly  more  likely  to  use  didactic  instruction  (traditional)  than  high-poverty  schools; 
otherwise,  there  is  no  significant  difference  between  high-  and  low-poverty  schools  in  types  of 
pedagogy  used. 


EXHIBIT  2.12 

Mean  Teacher  Use  of  Four  Pedagogical  Approaches  (Standard  Deviation), 
by  School  Level,  Subject,  and  Poverty  Level  (n=355) 


Traditional 

Nontr 

iditional 

illiii 

•TeScheiV: 
iCentered  \; 

Individual 

Seatwork 

Active 

i:OVientedtlll-lllll! 

School  Level 

Elementary 

47.13  (5.3CpJ 

1 

49.09  (5.92) 

> 

50.36  (4.72) 

51.38  (8.06# 

Middle 

50.47  (6.  l(j)* 

r 

49.41  (5.96) 

49.88  (7.95) 

50.16(8.00) 

High 

T 

53.33  (5.51) 

▼ 

51.33  (5.67) 

f 

49.62  (5.65) 

48.30  (9.07; 

Subject 

Mathematics 

50.53  (6.23) 

51.66  (5.34y 

i 

47.93  (5.70? 

50.23  (19.11) 

Science 

49.85  (6.14) 

48.27  (6.00)< 

* 

52.11  (5.347 

49.66  (8.35) 

Poverty  Level 

High  Poverty 

48.46  (6.03) 

49.98  (6.24) 

49.93  (6.36) 

50.60  (8.47) 

Low  Poverty 

51.20  (6.08) 

i 

49.96  (5.64) 

50.00  (5.66) 

49.59  (8.91) 

+ n=342 


Source : Longitudinal  Teacher  Survey,  Spring  1998. 

How  to  read  this  exhibit : The  group  of  three  cells  at  the  upper  left  of  the  table  (scale  score  means  for  teacher-centered  instruction  by 
school  level)  shows  that  high  school  teachers  use  significantly  more  teacher-centered  instruction  than  middle  school  teachers  and 
elementary  school  teachers,  and  middle  school  teachers  use  significantly  more  teacher-centered  instruction  than  elementary  school 
teachers.  The  table  should  be  read  by  columns,  focusing  on  each  pedagogical  strategy  separately. 

Note:  Results  in  table  are  scale  scores,  with  a mean  of  50  for  the  teachers  in  the  sample.  The  arrows  indicate  significant  differences 
between  groups  (p<.05),  with  the  head  of  the  arrow  showing  the  direction  of  the  difference. 


The  Relationship  between  Pedagogy  and  Content 

Both  pedagogy  and  content  are  critical  for  successful  instruction,  and  teachers  who  tend  to 
have  high  standards  for  one  also  have  high  standards  for  the  other.  In  this  section,  we  examine  the 
relationship  between  teachers'  pedagogical  approaches  and  the  content  of  their  instruction  to 
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determine  empirically  whether  certain  pedagogical  approaches  are  consistent  with  high  standards  for 
content.  We  explore  this  premise  by  correlating  teachers'  pedagogical  approaches  with  the  content  of 
their  instruction.  Teachers  whose  instruction  is  well-aligned  with  NAEP  tend  to  emphasize  seatwork 
and  discussion,  and  de-emphasize  active,  project-centered  instruction  (see  Exhibit  2.12). 

We  also  focused  on  one  of  the  two  components  of  content:  teachers'  performance  goals  for 
students  (see  Exhibit  2.13).  It  may  be  that  teachers  expect  a certain  level  of  performance  from  their 
students,  emphasizing  more  concrete  or  more  abstract  performance  goals,  and  that  they  choose 
pedagogical  approaches  to  help  their  students  reach  those  types  of  performance  goals.  We  examined 
the  relationship  between  teachers'  performance  goals  for  students  and  the  pedagogical  approaches 
they  favor.  We  found  that  traditional,  didactic  instruction  emphasizes  the  more  concrete  performance 
goals,  such  as  memorizing  and  performing  procedures,  and  de-emphasizes  performance  goals  that 
involve  abstract  thinking,  such  as  generating  hypotheses,  collecting/analyzing/interpreting  data,  and 
making  connections.17 

EXHIBIT  2.13 

Correlations  between  Pedagogical  Approaches  and  Elements  of  Content  (n=355) 


Didactic 

Individual 

Active,  Project- 

Discussion- 

Seatwork 
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Oriented* 
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Index  of  Alignment 
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Memorize 
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Perform  Procedures 
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.25*** 
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_ 24*** 
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2g*** 

ns 
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_ 2 1 *** 

ns 

ns 

+ n=342 

* statistically  significant  at  p<.05 

**  statistically  significant  at  p<.01 

***  statistically  significant  at  p<.001 

ns  not  statistically  significant 


Source : Longitudinal  Teacher  Survey,  Spring  1998. 

How  to  read  this  exhibit : The  first  cell  shows  that  didactic  pedagogy  is  significantly  correlated  with  the  performance  goal  of  memorizing; 
in  other  words,  the  more  that  teachers  emphasize  memorizing,  the  more  that  they  present  or  lecture.  The  table  should  be  read  by  columns, 
focusing  on  each  pedagogical  strategy  separately.  


Nontraditional,  active  instruction,  on  the  other  hand,  emphasizes  complex  performance  goals 
including  generating  hypotheses,  collecting/analyzing/interpreting  data,  and  making  connections,  and 
de-emphasizes  more  concrete  performance  goals,  including  memorizing  and  performing  procedures. 
In  other  words,  teachers  who  would  like  their  students  to  develop  more  concrete  performance  goals. 


17  Contrary  to  expectations,  the  performance  goal  of  students'  understanding  concepts  is  related  to  traditional 
instruction.  It  may  be  that  teachers  see  this  performance  goal  as  concrete  (e.g.,  students  can  recite  a research  theory) 
rather  than  abstract  (e.g.,  students  can  explain  a research  theory). 
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such  as  memorizing,  tend  to  use  traditional,  didactic  pedagogy  and  teachers  who  would  like  their 
students  to  develop  abstract  performance  goals,  such  as  making  connections,  tend  to  use  active, 
project-centered  pedagogy.  This  finding  is  consistent  with  the  literature  on  pedagogy,  as  well  as 
national  pedagogical  standards  for  mathematics  and  science,  which  advocate  using  active  instruction 
to  help  students  develop  complex  thinking  skills. 

Summary:  Pedagogy  and  High  Standards 

Traditional  and  nontraditional  teaching  strategies  can  be,  and  are,  part  of  instruction  that 
meets  high  standards.  Some  teachers  incorporate  seatwork  with  discussion,  and  the  results  indicate 
that  both  seatwork  and  discussion  are  related  to  high  content  standards.  Some,  but  not  all,  teachers 
incorporate  nontraditional  pedagogy  that  actively  involves  students  into  their  teaching.  Professional 
development  can  play  a key  role  in  helping  teachers  learn  to  integrate  traditional  and  nontraditional 
teaching  strategies,  and  help  them  improve  the  quality  of  student-centered  instruction.  For  example, 
in-class  observations  and  feedback,  following  professional  development  experiences,  can  help 
teachers  understand  whether  they  are  using  the  tools  of  active  instruction  on  a superficial  level  (e.g., 
discussions  that  are  geared  to  identifying  the  "correct"  answer)  or  pushing  students  toward  more 
complex  understanding  (e.g.,  discussions  that  encourage  students  to  surface  underlying  concepts). 
Later  chapters  in  this  report  will  examine  the  extent  to  which  Eisenhower-assisted  activities  offer 
active  learning  opportunities,  as  well  as  other  characteristics  of  high-quality  professional 
development.  We  also  will  examine  how  the  design  and  characteristics  of  Eisenhower-assisted 
activities  are  related  to  teacher  outcomes. 


In  this  chapter,  we  have  established  a baseline  understanding  of  the  approaches  to  instruction 
reported  by  teachers  in  the  Longitudinal  Study  of  Teacher  Change.  By  comparing  the  content 
covered  by  teachers  in  the  sample  with  content  included  in  NAEP  items,  critical  areas  have  been 
identified  where  professional  development  could  contribute  to  teaching  practice. 

In  addition,  we  have  examined  the  degree  to  which  teachers  in  the  sample  rely  on  traditional 
and  non-traditional  pedagogical  methods.  Prior  research  indicates  that  at  least  two  elements  of 
instruction  matter  for  student  learning:  content  that  is  aligned  with  high  standards  and  pedagogy 
emphasizing  active  learning.  The  sampled  teachers  are  working  toward  both  elements  of  effective 
instruction,  but  are  not  there  yet. 


Summary  and  Conclusions 
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CHAPTER  3 


TEACHERS’  EXPERIENCES  IN  EISENHOWER-ASSISTED 
PROFESSIONAL  DEVELOPMENT  ACTIVITIES 


The  goal  of  professional  development  is  to  improve  teaching.  In  the  last  chapter,  we 
discussed  the  strengths  and  weaknesses  of  current  teaching  practice  in  mathematics  and  science,  and 
we  identified  areas  in  which  professional  development  appears  to  be  needed.  In  particular,  many 
teachers  do  not  appear  to  give  sufficient  attention  to  advanced  topics,  and  many  attempt  to  cover  too 
many  topics  in  too  little  depth.  In  addition,  although  many  teachers  have  begun  to  employ  active, 
discussion-oriented  teaching  strategies,  the  use  of  these  strategies  is  frequently  superficial.  We  now 
begin  to  examine  the  features  of  professional  development  that  are  associated  with  changes  in 
teaching. 

Title  II  of  the  Elementary  and  Secondary  Education  Act,  as  amended  by  the  Improving 
America’s  Schools  Act  of  1994,  states  as  its  purpose  to  increase  the  quality  of  professional 
development  and  provide  professional  development  to  teachers  of  at-risk  students.  The  legislation 
states  that  its  purpose  is  to  provide  teachers  with  access  to  “sustained  and  intensive  high-quality 
professional  development”  (Section  2002(1))  that  among  other  things  “is  of  sufficient  intensity  and 
duration  to  have  a positive  and  lasting  impact  on  the  teacher's  performance  in  the  classroom” 
(Section  2002(2)(E)).  The  legislation  also  seeks  to  provide  professional  development  that  “reflects 
recent  research  on  teaching  and  learning”  (2002(2)(B))  and  “includes  strong  academic  content  and 
pedagogical  components”  (2002(2)(C)). 

The  Eisenhower  Program  also  places  an  emphasis  on  providing  professional  development  to 
teachers  in  Title  I schools.  The  legislation  provides  that  LEAs  and  SEAs  describe  in  their 
applications  how  activities  will  address  the  needs  of  teachers  in  Title  I schools  (Section 
2205(b)(2)(E)  and  Section  2208(d)(1)(B))  and  includes  them  in  the  needs  assessment  process 
(Section  2208(b)(2)). 

In  this  chapter,  we  examine  the  extent  to  which  Eisenhower-assisted  activities  meet  these 
provisions  of  the  legislation.  We  also  examine  the  extent  to  which  supported  activities  reflect  the 
characteristics  of  high-quality  professional  development  that  have  emerged  in  the  literature.  Finally, 
we  report  on  the  extent  to  which  Eisenhower-assisted  activities  meet  the  standards  that  have  been  set 
in  the  performance  indicators  for  the  program  established  by  the  Department  of  Education,  in 
compliance  with  the  Government  Performance  and  Results  Act  (GPRA). 

Exhibit  3.0  shows  where  these  topics  fit  into  the  framework  of  the  entire  report. 
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EXHIBIT  3.0 


Conceptual  Framework  for  This  Evaluation 


The  purpose  of  this  chapter  is  to  identify  features  of  professional  development  that  might  be 
appropriately  incorporated  in  Eisenhower-assisted  activities  because  they  have  been  shown  to  be 
effective  in  improving  teaching  practice.  To  this  end,  we  first  describe  the  features  of  effective,  high- 
quality  professional  development  identified  in  the  literature.  Second,  we  present  results  from  our 
study  describing  the  extent  to  which  Eisenhower-assisted  activities  reflect  these  features.  Third,  we 
use  our  data  to  assess  the  extent  to  which  these  features  are  related  to  teachers’  reports  of 
improvement  in  teaching  practice.  Our  analyses  have  implications  for  federal  legislation  and 
regulations,  which  call  for  Eisenhower-assisted  activities  to  have  particular  qualities  and  features.  In 
effect,  this  study  assesses  the  effectiveness  and  appropriateness  of  the  current  legislation’s  language 
and  requirements,  given  our  findings  on  the  importance  of  particular  characteristics  of  professional 
development.  The  framework  that  we  use  to  model  the  features  and  impact  of  professional 
development  is  based  on  both  the  professional  development  literature  and  the  analysis  of  our  survey 
data.  .( -<i.  • ■ 

High-quality  Professional  Development 

Over  the  past  decade,  a considerable  literature  has  emerged  on  professional  development, 
teacher  learning,  and  teacher  change.1  The  research  literature  contains  a mix  of  large-  and  small- 
scale  studies,  including  intensive  case  studies  of  classroom  teaching,  evaluations  of  programs 
designed  to  improve  teaching  and  learning,  and  surveys  of  teachers  about  their  pre-service 


1 See  V.  Richardson  & Placier,  in  press,  for  a comprehensive  review  of  the  literature  on  teacher  learning  and 
professional  development.  ; v 
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preparation  and  in-service  professional  development  experiences.2  In  addition,  there  is  a large 
literature  describing  “best  practices”  in  professional  development,  drawing  on  expert  experiences. 
Despite  the  size  of  the  literature,  however,  relatively  little  systematic  research  has  been  conducted  on 
the  effects  of  professional  development  on  improvements  in  teaching  or  on  student  outcomes. 

Although  relatively  little  research  has  been  conducted  on  the  effects  of  alternative  forms  of 
professional  development,  the  research  that  has  been  conducted,  along  with  the  experience  of  expert 
practitioners,  provides  some  preliminary  guidance  about  the  characteristics  of  high-quality 
professional  development.  (See,  in  particular,  Loucks-Horsley  et  al.,  1998.)  Recently,  for  example, 
the  U.S.  Department  of  Education,  drawing  on  the  literature  on  teachers  and  teacher  learning, 
identified  six  features  of  “best  practice”  in  professional  development.  According  to  the  Department 
(1999b,  p.  63),  effective  professional  development: 

♦ Reflects  an  image  of  teaching  and  learning  that  embraces  high  standards  for  all 
students.  Those  who  provide  professional  development  must  have  a clear  image  of 
effective  classroom  learning  and  teaching.  Good  professional  development  is  grounded 
in  approaches  to  teaching  that  enable  all  students  to  achieve  high  standards,  and  makes 
these  approaches  real  and  accessible. 

♦ Focuses  on  deepening  teachers’  knowledge  of  content  and  how  students  learn 
specific  content.  Good  professional  development  develops  teachers’  skill  and 
knowledge  regarding  the  disciplines  that  they  teach  and  children’s  ways  of  learning  at 
different  ages  and  in  different  contexts.  It  is  rigorous,  relevant,  and  research-based. 

♦ Provides  extended,  in-depth  learning  opportunities  for  teachers.  Good  professional 
development  promotes  learning  through,  among  other  things,  modeling  the  methods  to  be 
used  with  students  and  showing  how  methods  are  adapted  for  different  types  of  students. 
Like  all  good  teaching,  such  experiences  build  on  existing  knowledge,  immerse  learners 
in  stimulating  processes,  allow  for  teamwork,  and  spread  out  over  time  to  permit  learners 
to  digest  new  ideas,  try  them  out,  and  re-gather  for  critical  feedback.  Professional 
learning  is  embedded  in  the  life  of  the  school  whenever  appropriate. 

♦ Supports  expanded  roles  for  teachers  as  leaders  and  colleagues.  Good  professional 
development  provides  opportunities  for  teachers  to  serve  as  mentors,  peer  coaches, 
leaders,  designers,  planners,  and  facilitators.  Such  roles  encourage  collegial  relationships 
in  a community  of  learners.  These,  in  turn,  could  entail  changes  in  allocations  of 
authority,  responsibility,  and  time  in  schools. 

♦ Links  to  an  educational  system’s  programs  and  standards.  To  strengthen  the 
effectiveness  of  schools,  professional  development  should  be  linked  to  other  federal,  state 
or  district  initiatives.  It  should  be  tied  to  relevant  curricula,  assessments,  and  standards. 

♦ Is  accountable  for  results.  Professional  development  should  be  regularly  evaluated  for 
its  impact  on  teaching  and  learning,  and  evaluation  results  should  be  used  to  support 
continuous  improvement. 


2 See,  for  example,  Cohen  (1990)  for  a recent  intensive  case  study  of  change  in  mathematics  teaching;  Carey  and 
Frechtling  (1997)  for  a program  evaluation  of  exemplary  professional  activities  in  science;  and  U.S.  Department  of 
Education  (1999a)  for  a national  survey  of  teachers  focused  on  teacher  preparation  and  qualifications. 


3-3 


Similar  characteristics  appear  in  other  recent  examinations  of  mathematics  and  science 
teaching.  For  example,  James  Hiebert,  in  a review  of  the  research  on  mathematics  teaching  and 
learning  conducted  for  the  National  Council  of  Teachers  of  Mathematics,  calls  attention  to  the 
importance  of  high  standards,  content  focus,  and  in-depth  learning  opportunities  for  teachers, 
although  he  uses  somewhat  different  language.  According  to  Hiebert, 

Research  on  teacher  learning  shows  that  fruitful  opportunities  to  learn  new  teaching  methods 
share  several  core  features:  (a)  ongoing  (measured  in  years)  collaboration  of  teachers  for 
purposes  of  planning  with  (b)  the  explicit  goal  of  improving  students’  achievement  of  clear 
learning  goals,  (c)  anchored  by  attention  to  students’  thinking,  the  curriculum,  and  pedagogy, 
with  (d)  access  to  alternative  ideas  and  methods  and  opportunities  to  observe  these  in  action 
and  to  reflect  on  the  reasons  for  their  effectiveness...  (1999,  p.15). 

Although  lists  of  characteristics  such  as  these  commonly  appear  in  the  literature  on  effective 
professional  development,  there  is  little  direct  evidence  on  the  extent  to  which  these  characteristics 
are  related  to  positive  outcomes  for  teachers  and  students.  Some  studies  conducted  over  the  past 
decade  suggest  that  professional  development  experiences  that  share  all  or  most  of  these 
characteristics  can  have  a substantial,  positive  influence  on  teachers’  classroom  practice  and  student 
achievement.3  Several  recent  studies  have  begun  to  examine  the  relative  importance  of  specific 
dimensions  or  characteristics  of  professional  development.  For  example,  a number  of  recent  studies 
suggest  that  the  intensity  or  duration  of  professional  development  is  related  to  the  depth  of  teacher 
change  (Shields,  Marsh,  & Adelman,  1998;  Weiss  et  al.,  1998).  Furthermore,  there  is  some 
indication  that  professional  development  that  focuses  on  specific  mathematics  and  science  content 
and  the  ways  students  learn  such  content  is  especially  helpful,  particularly  for  instruction  designed  to 
improve  students’  conceptual  understanding  (Cohen  & Hill,  1998;  Fennema  et  al.,  1996).  However, 
few  studies  have  explicitly  compared  the  effects  of  different  forms  of  professional  development  on 
teaching  and  learning.4 

Thus,  there  is  a clear  need  for  new,  systematic  research  on  the  effectiveness  of  alternative 
strategies  for  professional  development.  The  National  Research  Council,  for  example,  in  a review  of 
recent  research  on  the  cognitive  sciences,  teaching,  and  learning,  argues  that 

Research  studies  are  needed  to  determine  the  efficacy  of  various  types  of  professional 
development  activities,  including  pre-service  and  in-service  seminars,  workshops,  and 
summer  institutes.  Studies  should  include  professional  development  activities  that  are 
extended  over  time  and  across  broad  teacher  learning  communities  in  order  to  identify  the 
processes  and  mechanisms  that  contribute  to  the  development  of  teachers’  learning 
communities  (Bransford  et  al.,  1999,  p.  240). 

3 See,  for  example,  Fennema  et  al.  (1996),  an  experimental  study  examining  the  effects  of  Cognitively  Guided 
Instruction,  an  intervention  in  elementary  school  mathematics;  Wilson  and  Ball  (1991),  an  intensive  case  study  of 
two  teachers  who  participated  in  the  Summer  Math  program;  and  Cohen  and  Hill  (1998),  which  describes  the 
relationship  between  participation  in  professional  development,  teaching  practice,  and  student  achievement,  using 
survey  data  from  California.  See  Kennedy  (1998)  for  a review  of  available  randomized  studies  examining  the 
effects  of  teacher  professional  development  on  student  achievement  in  mathematics  and  science.  See  Shields, 
Marsh,  and  Adelman  (1998)  for  a recent  examination  of  the  effects  of  the  NSF  SSIs  on  classroom  practice  in 
mathematics  and  science;  and  Weiss  et  al.  (1998)  for  an  examination  of  the  effects  of  the  NSF  LSCs. 

Kennedy  (1998)  and  Cohen  and  Hill  (1998)  are  among  the  few  examples  of  studies  that  compare  the  relative 
effectiveness  of  different  forms  of  professional  development.  Both  studies  conclude  that  professional  development 
focused  on  the  teaching  and  learning  of  specific  mathematics  and  science  content  is  more  effective  than  more 
general  professional  development. 
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We  designed  the  evaluation  of  Eisenhower-assisted  professional  development  activities  to 
enable  us  to  examine  the  relationship  between  features  of  professional  development  that  have  been 
identified  in  the  literature  and  change  in  teachers’  knowledge  and  skills  and  classroom  teaching 
practices,  as  reported  by  teachers.  We  integrated  and  operationalized  the  ideas  in  the  literature  on 
“best  practices”  in  professional  development  to  create  a set  of  measures  or  scales  describing  the 
characteristics  of  Eisenhower-assisted  activities. 

In  our  analysis  of  the  characteristics  of  high-quality  professional  development,  we  begin  by 
focusing  on  what  we  term  “structural  features.”  These  are  characteristics  of  the  structure  or  design  of 
a professional  development  activity.  We  focus  in  particular  on  three  structural  features: 

♦ The  form  or  organization  of  the  activity — that  is,  whether  the  activity  is  organized  as  a 
reform  type,  such  as  a study  group,  teacher  network,  mentoring,  committee  or  task  force, 
internship,  individual  research  project,  or  teacher  research  center,  in  contrast  to  a 
traditional  workshop  or  conference; 

♦ the  duration  of  the  activity,  including  the  total  number  of  contact  hours  that  participants 
are  expected  to  spend  in  the  activity,  as  well  as  the  span  of  time  over  which  the  activity 
takes  place;  and 

♦ the  degree  to  which  the  activity  emphasizes  the  collective  participation  of  groups  of 
teachers  from  the  same  school,  department,  or  grade  level,  as  opposed  to  the  participation 
of  individual  teachers  from  many  schools. 

In  addition  to  these  structural  features,  we  focus  on  three  dimensions  of  the  substance  or  core 
of  the  professional  development  experience.  We  examine  three  “core  features”: 

♦ the  degree  to  which  the  activity  has  a content  focus — that  is,  the  degree  to  which  the 
activity  is  focused  on  improving  and  deepening  teachers’  content  knowledge  in 
mathematics  and  science; 

♦ the  extent  to  which  the  activity  offers  opportunities  for  active  learning,  such  as 
opportunities  for  teachers  to  become  actively  engaged  in  the  meaningful  analysis  of 
teaching  and  learning;  for  example,  by  reviewing  student  work  or  obtaining  feedback  on 
their  teaching;  and 

♦ the  degree  to  which  the  activity  promotes  coherence  in  teachers’  professional 
development,  by  incorporating  experiences  that  are  consistent  with  teachers’  goals, 
aligned  with  state  standards  and  assessments,  and  encouraging  of  continuing  professional 
communication  among  teachers. 

Through  our  study  of  the  literature  and  the  analysis  of  our  survey  data,  we  have  formulated  a 
model  characterizing  the  relationships  between  the  quality  of  professional  development,  defined  by 
the  structural  and  core  features,  and  their  role  in  improving  teacher  outcomes.  In  this  model,  the 
three  structural  features — type,  duration,  and  collective  participation — are  conditions  that  enable  or 
facilitate  desire  core  features  of  professional  development.  That  is,  they  set  the  parameters  or  context 
within  which  teacher  learning  takes  place.  The  three  core  features  of  the  professional  development 
activity — content  focus,  active  learning,  and  coherence — are  factors  that  characterize  the  processes 
that  occur  during  a professional  development  experience.  By  focusing  on  specific  mathematics  and 
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science  content,  by  engaging  teachers  in  active  work,  and  by  fostering  a coherent  set  of  learning 
experiences,  we  hypothesize  that  a professional  development  activity  is  likely  to  enhance  the 
knowledge  and  skills  and  improve  the  classroom  teaching  practice  of  participating  teachers.  This,  of 
course,  serves  the  ultimate  goal  of  improved  student  learning. 

Data  Sources 

For  the  analyses  in  this  chapter,  we  use  information  from  the  Teacher  Activity  Survey 
conducted  as  part  of  the  evaluation.  In  the  spring,  summer,  and  fall  of  1998,  we  surveyed  a 
nationally  representative  sample  of  teachers  who  had  attended  Eisenhower-assisted  activities  over  the 
period  from  July  1 through  December  31,  1997.5  We  carried  out  the  survey  by  drawing  a national 
probability  sample  of  districts  and  SAHE  grantees.  SAHE  grantees  include  institutions  of  higher 
education  (IHEs)  such  as  universities,  four-year  colleges,  or  two-year  colleges,  and  not-for-profits 
(NPOs),  which  are  organizations  such  as  zoos,  museums,  and  libraries.  Districts  were  sampled  in 
proportion  to  the  number  of  teachers  in  the  district,  and  SAHE  grantees  were  sampled  in  proportion 
to  the  size  of  their  Eisenhower  grant,  based  on  the  logic  that  SAHE  grantees  with  larger  grants  would 
serve  larger  numbers  of  teachers.  For  each  district  and  SAHE  grantees  drawn  into  the  sample,  we 
collected  a complete  list  of  all  professional  development  activities  conducted  with  Eisenhower  funds 
over  the  period  from  July  through  December,  1997.  We  then  drew  a sample  of  two  activities  in  each 
district  or  SAHE  grantee,  with  the  probability  of  an  activity  being  selected  in  proportion  to  the 
number  of  teachers  attending  the  activity.  We  then  randomly  subsampled  two  teachers  who  attended 
each  activity.  We  received  responses  from  1,027  teachers,  representing  activities  supported  by 
Eisenhower  funds  in  358  districts  and  SAHE  grantees.  This  produced  an  overall  teacher  response 
rate  of  72  percent.6 

The  survey  asked  each  teacher  to  provide  detailed  information  about  the  specific  Eisenhower- 
assisted.  professional  development  activity  that  we  drew  in  our  sampling  process  and  that  led  the 
teacher  to  be  selected  for  our  sample.  Responses  are  self-reports  of  teacher  experiences  and 
behavior.7 

In  discussing  the  structural  features,  quality,  and  outcomes  of  Eisenhower-assisted 
professional  development  in  the  chapter,  we  also  include  information  from  our  In-Depth  Case 
Studies,  where  appropriate.  The  case  study  data  we  present  include  information  from  six  exploratory 
case  studies  that  we  conducted  in  the  spring  of  1997;  and  case  studies  of  ten  districts,  two  from  each 
of  five  states,  that  we  conducted  during  the  1997-1998  school  year.  We  chose  sites  to  allow  variation 

5 The  term  "Eisenhower-assisted  activities"  reflects  the  fact  that  district  Eisenhower  funds  can  support  professional 
development  activities  in  a number  of  ways.  Eisenhower  funds  may  be  used  to  support  all  costs  associated  with 
activities,  provided  that  these  activities  are  allowed  in  the  legislation  (See  Section  2210).  Alternatively,  Eisenhower 
funds  may  pay  for  only  some  of  the  allowable  costs  associated  with  an  activity.  This  is  a common  occurrence,  since 
the  legislation  encourages  cost  sharing  of  Eisenhower-assisted  professional  development  activities  with  those  funded 
by  other  programs  (Section  2209). 

6 See  Appendix  A for  a more  complete  discussion  of  the  sampling  plan  for  Teacher  Activity  Survey.  All  parameter 
estimates  reported  in  the  chapter  incorporate  weights  reflecting  the  sampling  plan.  Reported  p-values  and  the 
standard  errors  on  which  they  are  based,  however,  do  not  reflect  the  clustering,  stratification,  and  variance  in 
weights  incorporated  in  the  design.  Analyses  that  take  these  elements  of  the  complex  sample  design  into  account 
have  been  carried  out,  and  the  results  are  nearly  identical  to  those  reported  in  the  chapter. 

7 The  study  was  designed  to  maximize  our  capacity  to  provide  a national  description  of  Eisenhower-assisted 
activities  and  to  assess  the  relationship  between  characteristics  of  activities  and  teachers’  self-reported  change  in 
knowledge  and  skills  and  teaching  practices.  The  study  was  not  designed  to  allow  us  to  examine  the  relationship 
between  professional  development  and  student  achievement. 
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according  to  efforts  in  state-level  reform,  the  district’s  approach  to  professional  development,  and 
demographic  and  geographic  characteristics.8 

Organization  of  Chapter 

In  the  sections  that  follow,  we  draw  on  our  national  sample  of  teachers,  supplemented  with 
data  from  the  case  studies,  to  examine  the  extent  to  which  Eisenhower-assisted  activities  incorporate 
practices  that  reflect  high-quality,  in  terms  of  the  three  structural  features  and  the  three  core  features 
of  professional  development.9  Then,  we  use  the  data  to  examine  the  relative  strength  of  the 
relationship  between  these  quality  features  and  teachers’  reported  improvement  in  knowledge  and 
skills  and  teaching  practices.10 

In  many  of  the  exhibits  in  this  chapter,  we  contrast  teachers’  experiences  in  district  and 
SAHE-grantee  activities.  These  two  components  of  the  program  operate  in  somewhat  different 
organizational  contexts.  In  particular,  as  we  discuss  in  more  detail  in  chapters  4,  5,  and  6,  districts 
receive  Eisenhower  funds  through  an  allocation  formula,  while  SAHE  grantees  compete  for  funds. 

In  addition,  districts  generally  offer  a range  of  Eisenhower-assisted  activities  serving  a relatively 
large  number  of  teachers,  while  SAHE  grantees  tend  to  offer  a few  activities  focused  on  a small 
number  of  teachers.  As  we  discuss  further  in  later  chapters,  these  differences  in  organizational 
context  may  help  explain  differences  in  teacher  experiences  in  these  two  components  of  the  program. 

This  chapter  is  organized  in  five  sections.  The  first  section  examines  the  structural  features 
of  Eisenhower-assisted  professional  development  activities,  and  the  second  examines  the  core 
features  of  teachers’  professional  development  experiences  in  the  activities  in  which  they 
participated.  The  third  section  examines  the  extent  to  which  teachers  report  that  participation 
enhanced  their  knowledge  and  skills  and  improved  their  teaching  practices,  and  the  fourth  examines 
the  relationship  between  structural  features,  core  features,  and  teachers’  reported  enhancement  of 
knowledge,  skills,  and  teaching  practices.  The  fifth  section  examines  the  extent  to  which  teachers  in 
Eisenhower-assisted  activities  teach  in  high-poverty  schools,  and  the  final  section  summarizes  our 
evidence  from  participating  teachers  and  draws  some  initial  conclusions  about  the  Eisenhower 
program. 


8 See  Appendix  B for  a more  complete  discussion  of  the  design  of  the  In-Depth  Case  Studies. 

9 Results  for  some  analyses  reported  in  this  chapter  were  reported  earlier  in  U.S.  Department  of  Education  (1999b). 
The  earlier  results  differ  from  results  in  this  report  because  they  were  preliminary,  unweighted,  and  did  not  include 
the  full  sample  of  teachers  and  districts.  Results  are  considered  to  be  statistically  significant  if  the  p- value  is  .05  or 
smaller. 

10  The  survey  results  in  this  chapter  are  based  on  teachers’  self-reports  on  change  in  knowledge,  skills,  and  teaching 
practices.  We  are  also  collecting  longitudinal  data  on  a sample  of  teachers  as  part  of  the  evaluation.  These 
longitudinal  data  will  provide  stronger  evidence  of  change  in  classroom  teaching,  and  they  will  permit  us  to  test  the 
relationships  that  we  discuss  in  this  chapter.  Analyses  of  the  longitudinal  data  will  appear  in  the  third  evaluation 
report,  scheduled  to  be  available  in  the  spring  of  2000. 
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Structural  Features 


Section  Findings 

♦ Most  teachers  who  participate  in  Eisenhower-assisted  activities  report  that  the  activities 
take  the  form  of  workshops  or  conferences,  for  both  districts  and  SAHE  grantees;  but 
some  teachers  report  participating  in  reform  types  of  activities,  such  as  study  groups, 
mentoring,  or  networks. 

♦ Activities  supported  under  the  SAHE  component  of  the  program  are  of  substantial 
duration,  often  lasting  more  than  80  hours  and  spanning  a period  of  several  months  or 
more.  Activities  supported  under  the  district  component  tend  to  be  of  shorter  duration, 
although  the  duration  appears  to  have  increased  since  the  previous  Eisenhower 

evaluation. 

♦ Although  some  Eisenhower-assisted  activities  are  designed  for  groups  of  teachers  from 
the  same  school,  department,  or  grade  level,  most  are  designed  for  individual  teachers 

While  the  authorizing  legislation  for  the  Eisenhower  program  does  not  specify  the  structure 
of  supported  activities  in  detail,  the  language  included  in  the  legislation  nevertheless  has  important 
implications  for  the  structure.  In  particular,  the  legislation  includes  language  on  both  duration  and 
collective  participation,  two  of  the  three  structural  features  on  which  we  focus.  With  respect  to 
duration,  the  legislation  includes  an  intent  to  support  professional  development  that  “is  of  sufficient 
intensity  and  duration  to  have  a positive  and  lasting  effect  on  the  teacher’s  performance  in  the 
classroom”  (Section  2002(2)(E)).  Although  the  legislation  does  not  specify  a required  minimum 
duration,  supported  activities  should  be  long  enough  to  have  “lasting  effects.” 

In  addition,  the  legislation  includes  language  pertaining  to  the  third  structural  feature — the 
extent  to  which  activities  encourage  the  collective  participation  of  groups  of  teachers  from  the  same 
school,  department,  or  grade  level.  The  legislation  stipulates  that  LEAs  should  use  at  least  80  percent 
of  Eisenhower  funds  for  professional  development  for  staff  of  “individual  schools”  that  “is 
determined  by  such  teachers  and  staff’  and  “to  the  extent  practicable,  takes  place  at  the  individual 
school  site”  (Section  2210(a)(l)(A)(B)).  The  law  also  suggests  that  funds  may  be  used  for 
“professional  development  for  teams  of  teachers”  (Section  2210(b)(3)(A)).  These  provisions,  taken 
together,  appear  to  encourage  activities  that  are  designed  for  collective  participation. 

In  the  paragraphs  that  follow,  we  examine  the  three  structural  features  of  Eisenhower-assisted 
professional  development  activities  in  more  detail,  relying  on  our  data  from  a nationally 
representative  sample  of  teachers  who  participated  in  Eisenhower-assisted  activities  during  the  1997- 
98  year.  We  begin  by  examining  the  type  or  organizational  form;  we  then  consider  the  duration,  and 
then  the  extent  to  which  supported  activities  encourage  collective  participation. 

Type  of  Activity 

Undoubtedly  the  most  common  type  or  form  of  professional  development,  and  the  form  most 
criticized  in  the  literature,  is  the  “workshop.”  A workshop  is  a structured  approach  to  professional 
development  that  occurs  outside  the  teacher’s  own  classroom.  It  generally  involves  a leader  or 
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leaders  with  special  expertise  and  participants  who  attend  sessions  at  scheduled  times — often  after 
school,  on  the  weekend,  or  during  the  summer  (Loucks-Horsley  et  al .,  1998,  pp.  42-43).  Institutes, 
courses,  and  conferences  are  other  traditional  forms  of  professional  development  that  share  many  of 
the  features  of  workshops,  in  that  they  tend  to  take  place  outside  of  the  teacher’s  school  or  classroom; 
they  involve  a leader  or  leaders  with  special  expertise,  and  participants  who  attend  at  scheduled 
times. 


While  traditional  forms  of  professional  development,  such  as  workshops,  institutes,  courses, 
and  conferences,  are  quite  common,  they  are  widely  criticized  as  being  ineffective  in  providing 
teachers  with  sufficient  time,  activities,  and  content  necessary  for  increasing  teacher’s  knowledge 
and  fostering  meaningful  changes  in  their  classroom  practice  (Loucks-Horsley  et  al.,  1998).  As  a 
result,  there  is  growing  interest  in  other  “reform”  types  of  professional  development,  such  as  study 
groups  or  mentoring  and  coaching.  These  reform  types  differ  from  traditional  professional 
development  in  several  respects.  In  particular,  reform  activities  often  take  place  during  the  regular 
school  day.  In  fact,  some  reform  activities,  such  as  mentoring  and  coaching,  take  place,  at  least  in 
part,  during  the  process  of  classroom  instruction  or  during  regularly  scheduled  teacher  planning  time. 
By  locating  opportunities  for  professional  development  as  part  of  a teacher’s  regular  work  day, 
reform  types  of  professional  development  may  be  more  likely  than  traditional  forms  to  make 
connections  with  classroom  teaching,  and  they  may  be  easier  to  sustain  over  time. 

In  addition,  reform  types  of  activities  may  be  more  responsive  to  how  teachers  learn  (Ball, 
1996),  and  may  have  more  of  an  influence  on  changing  teaching  practice  (Darling-Hammond,  1995; 
1996;  Hargreaves  & Fullan,  1992;  Little,  1993;  Richardson,  1994;  Sparks  & Loucks-Horsley,  1989; 
Stiles,  Loucks-Horsley,  & Hewson,  1996).  Further,  Darling-Hammond  argues  that  these  activities 
may  be  more  responsive  to  teachers’  needs  and  goals: 

Some  schools  have  begun  to  create  new  models  of  induction  and  ongoing  professional 
development  for  teachers  and  principals.  They  feature  mentoring  for  beginners  and  veterans, 
peer  observation  and  coaching,  local  study  groups  and  networks  for  developing  teaching 
within  specific  subject  matter  areas  (like  the  National  Writing  Project  or  the  Urban 
Mathematics  Collaboratives),  teacher  academies  that  offer  ongoing  seminars  and  courses  of 
study  tied  to  practice,  and  school-university  partnerships  that  sponsor  collaborative  research, 
interschool  visitations,  and  a variety  of  formal  and  informal  learning  opportunities  developed 
in  response  to  teachers’  and  principals’  felt  needs  (Darling-Hammond,  1997b,  p.  325). 

In  our  survey  of  teachers,  we  asked  each  teacher  to  describe  the  specified  Eisenhower- 
assisted  activity  in  which  the  teacher  participated,  and,  as  part  of  the  description,  we  asked  the 
teacher  to  specify  the  type  of  activity,  using  the  following  set  of  categories: 1 1 

♦ Within-district  workshops  or  institutes , focused  on  a specific  topic,  provided  by  or  within 
the  district. 

♦ Courses  for  college  credit. 

♦ Out-of-district  workshops  and  institutes,  focused  on  a specific  topic,  provided  outside  of 
the  district. 


11  The  categories  draw  in  part  on  Loucks-Horsley  et  al.  (1998). 
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♦ Out-of-district  conferences,  provided  by  professional  organizations,  regional  centers,  the 
state  department  of  education,  etc. 

♦ Teacher  study  groups  that  meet  regularly,  in  face-to-face  meetings,  to  further  teacher 
knowledge  in  their  disciplines  or  of  pedagogical  approaches. 

♦ Teacher  collaborates  or  networks,  connecting  teachers  regionally,  state-wide, 
nationally,  or  internationally  (this  does  not  include  activities  described  in  the  first  three 
bullets). 

♦ Committees  or  taskforces  focusing  on  curriculum,  instruction,  or  student  assessment. 

♦ Receiving  mentoring,  coaching,  lead  teaching,  or  observation,  in  a one-on-one  situation, 
usually  in  the  classroom. 

♦ Immersion  or  internship  activities,  in  which  a teacher  spends  a concentrated  period  of 
time  working  in  a lab  or  industrial  setting  with  professionals  in  his  or  her  subject  area. 

♦ Teacher  resource  centers,  which  provide  professional  development  materials  and  are 
staffed  by  a lead  or  resource  teacher. 

The  first  four  types  of  activities  (within-district  workshops,  courses  for  college  credit,  out-of- 
district  workshops,  and  out-of-district  conferences)  are  traditional  in  form;  the  remaining  types  of 
activities  (teacher  study  groups,  teacher  collaborative  or  networks,  committees,  mentoring, 
internships,  and  resource  centers)  are  reform  activities.12 

Exhibit  3.1,  below,  displays  information  from  the  teacher  survey  on  the  types  of  professional 
development  activities  supported  by  Eisenhower  funds.  The  data  indicate  that  most  Eisenhower- 
assisted  activities  are  traditional  in  form.  Overall,  79  percent  of  teachers  participating  in  district 
Eisenhower-assisted  activities  participated  in  traditional  types  of  activities,  including  52  percent  in 
in-district  workshops,  4 percent  in  college  courses,  15  percent  in  out-of-district  workshops  or 
institutes,  and  8 percent  in  conferences.  Similarly,  74  percent  of  teachers  participating  in  SAHE- 
grantee  activities  participated  in  traditional  types.  Some  teachers  report  that  the  activities  in  which 
they  participated  were  reform  types,  including  collaboratives  and  networks,  internships,  mentoring, 
resource  centers,  committees  and  task  forces,  and  study  groups,  but  the  overall  percent  of  teachers 
participating  in  reform  activities  is  relatively  small. 


12  The  survey  included  a final  category,  “other  organized  forms  of  professional  development,”  and  asked  the  teacher 
to  describe  the  form.  We  reclassified  all  responses  into  one  of  the  10  forms  listed. 
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EXHIBIT  3.1 

Percent  of  Teachers  Reporting  Participation  in  Traditional  and  Reform  Types  of 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=775,  SAHE  Grantee  n=244)13 


© 

f & 
r >• 

O •- 

a > 
© *- 
k.  o 

O)  <° 
JZ 

© o 
•C  ro 

£ • 

© £ 
**  — 

«*-  c 
o o 

c <0 
© a 


® r 

O-  m 


100%-. 
90%  - 
80% 
70%  - 
60% - 
50% 
40%  - 
30% - 
20% 
10%- 
0% 


Traditional 


b 

— 

15% 

•f 

4% 

• 

KB 

0% 


Reform 


i% 


3% 


■ Districts 
□ SAHE  Grantees 


2% 


1% 


3% 


1% 


2% 


I i *- 

In- O imtrict  C oil aga  Court#  Out- ol- District  Out-ol-Oiatriet  Out- of- District  Study  Group  Mentoring  Committee  or  Internship  Reeource 

Workshopor  Workahopor  Contaranca  C ottaborativa or  Task  Forca  Center 

Inatituta  InsUtuto  Network 


Activity  type 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998 

How  to  read  this  exhibit : The  first  bar  shows  that  52  percent  of  the  teachers  who  participated  in  district  activities  report  that  they  attended  an 
in-district  workshop  or  institute.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each  category. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


About  22  percent  of  teachers  participating  in  district  activities  and  26  percent  of  teachers 
participating  in  SAHE-grantee  activities  are  in  reform  types  of  activities,  including  collaboratives  and 
networks,  mentoring,  and  study  groups.  The  percent  of  teachers  in  such  activities  appears  to  have 
risen  since  the  previous  evaluation  of  the  Eisenhower  program,  conducted  in  1988-89  (Knapp  et  al., 
1991).  Nevertheless,  workshops  and  other  traditional  forms  continue  to  be  the  predominate  types  of 
professional  development  supported  with  Eisenhower  funds.14  In  Chapters  4,  5,  and  6,  we  explore 
some  of  the  reasons  for  the  continued  reliance  on  traditional  types  of  learning  opportunities. 

The  type  of  an  activity  may  set  the  context  for  many  other  features  of  the  activity’s  structure 
and  substance.  Because  reform  activities  such  as  study  groups  and  mentoring  often  take  place  during 
the  regular  school  day,  they  may  enable  activities  of  longer  duration  than  traditional  activities;  and 


1 ^ 

' Due  to  missing  data  caused  by  teacher  nonresponse  to  particular  survey  questions,  the  number  of  teacher 
responses  varies  across  survey  items. 

The  reported  results  concern  the  percent  of  teachers  in  reform  activities.  The  percent  of  Eisenhower  funds  spent 
on  reform  activities  may  be  higher,  if  reform  activities  cost  more  per  teacher  than  traditional  activities. 
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they  may  make  it  easier  to  encourage  the  collective  participation  of  groups  of  teachers  from  the  same 
school  or  department.  Given  the  potential  importance  of  activity  type  as  a key  structural  feature,  we 
contrast  traditional  and  reform  activities  in  reporting  our  results  throughout  the  chapter. 

Duration  of  Activity 

Almost  all  of  the  recent  literature  on  teacher  learning  and  professional  development  calls  for 
professional  development  that  is  sustained  over  time.  The  duration  of  professional  development 
activities  is  expected  to  be  important  for  two  reasons.  First,  longer  activities  are  more  likely  to 
provide  an  opportunity  for  in-depth  discussion  of  mathematics  and  science  content,  student 
conceptions  and  misconceptions,  and  pedagogical  strategies.  Second,  activities  that  extend  over  time 
are  more  likely  to  allow  teachers  to  try  out  new  practices  in  the  classroom  and  obtain  feedback  on 
their  teaching. 

Project  Science,  an  Eisenhower- assisted  professional  development  activity  in  Middle  City, 
Wisconsin,  illustrates  an  activity  that  extends  throughout  a full  year.  The  goal  of  Project  Science  is 
to  involve  teams  of  teachers  from  different  schools  in  establishing  the  capacity  to  develop  and 
administer  performance  assessments.  The  project  included  an  institute  in  the  summer,  as  well  as 
two-hour  monthly  meetings  over  the  school  year,  from  September  until  May.  In  addition,  during  the 
year,  school  networking  sessions  focused  on  curriculum  and  learning  issues  and  on  the  development 
of  assessment  instruments.  During  these  meetings,  teachers  shared  progress,  difficulties,  ideas, 
issues,  and  needs. 

To  develop  a national  estimate  of  the  duration  of  professional  development  supported  by  the 
Eisenhower  program,  we  included  items  on  our  Teacher  Activity  Survey  asking  about  two  aspects  of 
duration:  the  total  number  of  contact  hours  spent  in  the  professional  development  activity,  including 
all  components  of  the  activity  that  were  held  during  the  one-year  period  from  July  1,  1997,  through 
June  30,  1998;  and  the  span  or  period  of  time,  in  days,  weeks,  and  months,  over  which  the  activity 
was  spread.15  Exhibit  3.2  displays  our  results  for  contact  hours.  The  results  indicate  that 
Eisenhower-assisted  activities  vary  widely  in  the  amount  of  time  involved. 

In  the  graph,  each  dot  represents  one  teacher.  If  only  one  teacher  in  our  sample  attended  an 
activity  of  a particular  duration  (e.g.,  140  hours),  the  teacher  appears  as  a single  dot.  If  more  than 
one  teacher  in  our  sample  attended  an  activity  of  a particular  duration  (e.g.,  20  hours),  the  teachers 
are  displayed  in  a horizontal  line.  The  length  of  the  line  is  proportional  to  the  number  of  teachers. 
For  example,  the  graph  indicates  that  more  district  teachers  attended  activities  lasting  20  hours  than 
lasting  30  hours. 

The  number  appearing  on  the  right  of  each  distribution  is  the  average  number  of  hours  of 
instruction  for  that  particular  group  of  teachers.  Overall,  district  activities  last  an  average  of  25 
hours.  SAHE-grantee  activities  last  considerably  longer  than  district  activities.  The  average  length 
of  SAHE-grantee  activities  is  51  hours — nearly  twice  as  long  as  district  activities. 


15  Teachers  who  completed  their  surveys  before  the  end  of  the  1997-98  school  year  were  asked  to  estimate  the 
number  of  additional  hours  the  activity  would  last  during  the  remaining  months  of  the  school  year. 
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EXHIBIT  3.2 


Contact  Hours  Provided  by  Eisenhower-assisted  Professional  Development 
Activities,  as  Reported  by  Teachers  (District  n=767,  SAHE  Grantee  n=244) 


i 51 


* 52  _L  49 

• 


Overall  Traditional  Reform 

SAHE  Grantees 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit.  The  first  column  (District  “Overall”)  shows  that  on  average,  teachers  who  participated  in  district  activities  reported 
that  they  were  engaged  in  the  activity  for  25  hours.  Each  dot  represents  one  teacher.  If  more  than  one  teacher  reported  the  same  number  of 
hours,  the  teachers  are  displayed  in  a horizontal  line  with  length  proportional  to  the  number  of  teachers.  Each  column  represents  the 
distribution  for  a particular  group  of  teachers.  The  number  to  the  right  of  each  column  is  the  mean. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


While  district  activities  tend  to  be  shorter  than  SAHE-grantee  activities,  our  results  indicate 
that  the  median  duration  of  district  Eisenhower-assisted  activities  has  approximately  doubled  since 
the  last  evaluation  of  the  program  was  conducted  in  1988-89.  According  to  the  1988-89  evaluation, 
in  the  median  district,  activities  supported  by  Eisenhower  funds  lasted  an  average  of  6 hours,  which 
is  less  than  half  of  the  current  median  of  15  hours.16 

As  expected,  at  least  for  districts,  reform  activities  last  longer  than  traditional  types:  the 
average  length  of  reform  activities  is  35  hours,  compared  with  23  hours  for  traditional  activities.  For 
SAHE  grantees,  however,  there  is  essentially  no  difference  in  length  between  traditional  and  reform 
types.  In  part,  this  may  reflect  the  fact  that  many  traditional  SAHE-grantee  activities  are  college 
courses,  and  college  courses,  at  a minimum,  generally  last  at  least  three  hours  a week  for  ten  weeks. 
But  courses  do  not  fully  account  for  the  substantial  length  of  traditional  SAHE-grantee  activities; 
workshops  offered  by  grantees  tend  to  be  longer  than  those  offered  by  districts. 

One  way  to  assess  the  adequacy  of  the  number  of  contact  hours  of  professional  development 
provided  by  typical  Eisenhower-assisted  activities  is  to  compare  our  results  with  the  hours  provided 
in  widely  known  exemplary  professional  development  activities  in  mathematics  and  science.  One 
frequently  cited  professional  development  program  is  Cognitively  Guided  Instruction  (CGI),  which 
focuses  on  improving  teachers’  understanding  of  student  learning  in  elementary  arithmetic 


16  The  1988-89  evaluation  collected  data  on  duration  from  districts  rather  than  teachers,  so  a comparison  of  results 
from  the  1988-89  and  the  current  evaluation  should  be  interpreted  as  providing  an  indication  of  the  general 
magnitude  of  the  change  rather  than  a precise  numerical  estimate.  See  Knapp  et  al.  (1991,  p.  109). 
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(Carpenter  et  al.,  1989).  The  basis  of  the  intervention  is  an  80-hour  summer  institute,  with  modest 
year-long  follow-up.  Fennema  et  al.  (1996)  have  conducted  a number  of  careful  studies  that  show 
that  the  program  has  had  a substantial  positive  influence  on  participating  teachers’  teaching  practices 
and  on  student  achievement  as  measured  on  tests  of  mathematical  reasoning  and  problem  solving.  In 
a recent  study  of  exemplary  professional  development  activities  in  the  sciences  supported  by  NSF, 
ED,  and  other  federal  agencies,  the  activities  involved  lasted  from  a few  days  to  eight  weeks  (Carey 
and  Frechtling,  1997).  Other  studies  include  Cobb  et  al.  (1991)  and  Wood  and  Sellers  (1996),  who 
show  positive  effects  on  student  achievement  in  mathematical  reasoning,  based  on  a professional 
development  activity  lasting  about  150  hours;  and  the  Ohio  State  Systemic  Initiative  (Project 
Discovery),  which  produced  positive  changes  in  teaching  practice,  based  on  an  intensive  six-week 
(240  hours)  summer  program  (Supovitz,  1996). 

Many  SAHE-grantee  professional  development  activities  have  a duration  (in  hours) 
comparable  to  CGI,  and  this  suggests  that,  at  least  potentially,  these  activities  are  sufficiently 
intensive  to  help  teachers  achieve  worthwhile  change  in  teaching  practice.  Typically,  most  district 
Eisenhower- assisted  activities  are  substantially  shorter  than  exemplary  activities  in  mathematics  and 
science. 

In  addition  to  asking  about  the  hours  of  professional  development  provided,  we  also  asked 
about  the  span  of  the  activity,  or  the  period  over  which  the  sessions  or  components  of  the  activity 
were  spread.  Although  hours  and  span  are  correlated,  they  measure  different  aspects  of  duration, 
both  of  which  are  important  in  providing  teachers  with  sufficient  opportunities  for  in-depth  study, 
interaction,  and  reflection. 

The  results,  displayed  in  Exhibit  3.3,  show  that  some  Eisenhower-assisted  activities  take 
place  over  a one-day  period,  while  others  extend  over  a period  longer  than  one  month  and  some 
extend  six  months  or  more.  The  data  indicate  that,  overall,  about  27  percent  of  teachers  in  district 
Eisenhower-assisted  activities  report  a span  more  than  one  month,  while  56  percent  of  teachers  in 
IHE  activities  report  a span  more  than  one  month.  As  anticipated,  there  is  a substantial  difference  in 
span  between  traditional  and  reform  types.  For  districts,  23  percent  of  teachers  in  traditional 
activities  report  that  these  activities  last  more  than  a month,  while  41  percent  of  reform  activities 
exceed  one  month.  Similarly,  for  IHE/NPOs,  55  percent  of  teachers  in  traditional  activities  report 
that  they  last  longer  than  a month,  while  73  percent  of  teachers  in  reform  activities  report  that  they 
exceed  one  month. 

Many  Eisenhower-assisted  activities  take  place  over  a very  short  span  of  time.  Overall,  56 
percent  of  district  activities  span  a period  of  less  than  one  week  (less  than  a day  through  four  days) 
and  19  percent  of  SAHE-grantee  activities  span  a period  of  less  than  one  week. 
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EXHIBIT  3.3 


Time  Span  of  Eisenhower-assisted  Activities,  as  Reported  by  Teachers 
(District  n=766,  SAHE  Grantee  n=244) 
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Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  column  shows  that  2 percent  of  the  teachers  who  participated  in  district  activities  were  engaged  in  the 
activity  over  more  than  a year.  Each  dot  represents  one  teacher.  If  more  than  one  teacher  reported  the  same  span,  the  teachers  are  displayed 
in  a horizontal  line  with  length  proportional  to  the  number  of  teachers.  Each  column  represents  the  distribution  for  a particular  group  of 
teachers.  The  number  on  the  top  of  each  line  is  the  percent  of  teachers  participating  in  the  corresponding  time  span. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


Collective  Participation 

There  is  a growing  interest  in  professional  development  that  is  designed  for  groups  of 
teachers  from  the  same  school,  department,  or  grade  level.  Although  professional  development 
designed  for  groups  of  teachers  is  sometimes  thought  to  reinforce  the  status  quo,  it  also  has  a number 
of  potential  advantages.  First,  teachers  who  work  together  are  more  likely  to  have  the  opportunity  to 
discuss  concepts,  skills,  and  problems  that  arise  during  their  professional  development  experiences. 
Second,  teachers  who  are  from  the  same  school,  department,  or  grade  are  likely  to  share  common 
curriculum  materials,  course  offerings,  and  assessment  requirements.  By  engaging  in  joint 
professional  development,  they  may  be  able  to  integrate  what  they  learn  with  other  aspects  of  their 
instructional  context. 

Finally,  by  focusing  on  a group  of  teachers  from  the  same  school,  professional  development 
may  help  sustain  changes  in  practice  over  time,  as  some  teachers  leave  the  school’s  teaching  force 
and  other  new  teachers  join  the  faculty.  Professional  development  may  help  contribute  to  a shared 
professional  culture,  in  which  teachers  in  a school  or  teachers  who  teach  the  same  grade  or  subject 
develop  a common  understanding  of  instructional  goals,  methods,  problems,  and  solutions.  (See,  for 
example,  Talbert  & McLaughlin,  1993.)  Collective  participation  in  the  same  activity  can  provide  a 
forum  for  debate  and  improving  understanding,  which  increases  teachers’  capacity  to  grow  (Ball, 
1996). 


Knapp,  in  a 1997  review  of  the  effects  of  systemic  reform  in  mathematics  and  science,  gives 
particular  attention  to  the  importance  of  viewing  change  in  classroom  teaching  as  a problem  of 
organizational  as  well  as  individual  learning: 

Where  and  how  does  individual  learning  about  challenging  science  instruction,  for  example, 
get  deposited  in  organizational  routines?  How  does  a culture  supportive  of  science 
instruction  take  root  in  a school  and  sustain  itself  beyond  the  original  cast  of  characters  who 
helped  set  in  motion  a shift  in  organizational  culture?  (p.  258) 

Little  research  is  available  on  the  effects  of  collective  approaches  to  professional 
development,  but  there  is  some  evidence  that  it  can  be  effective  in  changing  teaching  practice. 
Newmann  and  associates,  in  a study  of  24  “restructuring  schools,”  note  that,  in  the  more  successful 
schools: 

Professional  development  tended  to  be  focused  on  groups  of  teachers  within  the  school  or  the 
faculty  as  a whole.  Making  use  of  internal  as  well  as  external  expertise,  staff  development 
activities  took  advantage  of  local  skills  and  sharing  of  effective  practice.  Including  internal 
experts  as  staff  developers  reinforced  teachers’  sense  of  commitment  to  their  school’s  goals 
(Newmann  et  al.,  1996). 

Maple  City,  Ohio,  one  of  our  case-study  districts,  illustrates  an  Eisenhower-assisted  activity 
that  encouraged  collective  participation  similar  in  form  to  the  learning  opportunities  Newmann  et  al. 
describe.  The  district  offers  several  subject-specific,  half-  to  full-day  Eisenhower-assisted  inservices 
by  grade  level.  The  inservices  are  led  by  the  math  and  science  coordinators  or  by  someone  outside 
the  district.  At  the  inservices,  teachers  share  information  and  instructional  practices,  review 
instructional  materials,  and  engage  in  activities  to  improve  their  skills.  They  are  hands-on  and 
support  an  integrated  approach  to  instruction,  including  an  emphasis  on  bridging  instruction  across 
contiguous  grade  levels.  Teacher-Leaders  help  teachers  implement  new  practices  by  serving  as 
mentors  in  their  classrooms.  They  plan  with  them,  team-teach,  and  gather  necessary  supplies.  In 
addition,  the  district  provides  teachers  with  time  during  the  school  day  to  get  together  and  discuss 
what  works  and  what  does  not. 

To  assess  the  prevalence  of  collective  participation  in  Eisenhower-assisted  activities,  we 
asked  each  teacher  in  our  national  sample  to  indicate  whether  the  activity  in  which  the  teacher 
participated  was  designed  for  all  teachers  in  a school  or  set  of  schools,  or  all  teachers  in  the  teacher’s 
department  or  grade  level.17  The  results,  displayed  in  Exhibit  3.4,  show  that  about  20  percent  of 
teachers  in  district  Eisenhower  activities  report  that  the  activity  was  designed  for  all  teachers  in  a 
department  or  grade  level,  and  19  percent  report  that  the  activity  was  for  all  teachers  in  a school. 
Fewer  teachers  in  SAHE-grantee  activities  report  either  form  of  collective  participation:  seven 
percent  indicate  that  their  activity  was  designed  for  all  teachers  in  a school,  and  1 1 percent  for  all 
teachers  in  a department  or  grade  level. 


17  Teachers  were  also  given  the  following  options:  teachers  as  individuals,  teachers  as  representatives  of  their 
departments,  grade  level,  or  schools,  and  other  configurations.  Teachers  could  check  all  that  applied. 


EXHIBIT  3.4 


Percent  of  Teachers  Reporting  School-level,  and  Department  or  Grade-level 
Participation  in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 
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S Districts 
□ SAHE  Grantees 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  20  percent  of  the  teachers  who  participated  in  district  activities  report  that  all  teachers  in 
department  or  grade-level  groupings  participated  in  the  professional  development  activity.  Each  bar  and  the  number  on  the  top  of  it  represent 
the  percent  of  teachers  for  each  category. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


Because  reform  activities  frequently  occur  during  the  school  day,  and  often  involve  work  in 
the  classroom,  we  had  anticipated  that  reform  activities  would  be  more  likely  to  be  designed  to 
promote  collective  participation  than  traditional  forms.  Our  data,  however,  show  no  difference 
between  traditional  and  reform  activities  in  collective  participation,  for  either  districts  or  IHE/NPOs. 
This  result  is  consistent  with  data  from  some  of  our  case  sites.  In  several  sites  with  mentors  and 
coaches,  for  example,  the  mentors  moved  across  schools  to  work  with  individual  teachers.  We  also 
observed  study  groups  whose  membership  crossed  school  and  department  boundaries. 

By  and  large,  most  Eisenhower-assisted  activities  are  designed  for  teachers  as  individuals. 
This  pattern  is  consistent  with  other  data  on  professional  development,  which  shows  that  collective 
participation  is  relatively  uncommon — indeed,  less  common  overall  than  our  data  indicate  is  the  case 
for  Eisenhower-assisted  activities.  For  example,  according  to  Shields,  Marsh,  and  Adelman  (1998), 
very  few  State  Systemic  Initiatives  (SSIs)  have  focused  their  professional  development  efforts  on 
whole  schools  or  departments,  and  those  that  did  focused  on  just  a few  sites.  As  Shields,  Marsh,  and 
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Adelman  argue,  one  reason  for  this  may  be  the  resources  required:  providing  sustained,  intensive 
professional  development  for  all  teachers  in  a school  may  require  more  resources  than  are  available. 
Programs  thus  face  a tradeoff  between  providing  longer-term  professional  development  for  some 
teachers,  or  focusing  on  whole  schools.18 

Summary:  Structural  Characteristics 

Overall,  our  data  from  a national  sample  of  teachers  who  have  attended  Eisenhower-assisted 
activities  show  that  most  teachers  participating  in  Eisenhower-assisted  activities  are  in  activities  that 
are  traditional  in  form,  although  about  20  percent  of  teachers  in  district  activities  and  one-quarter  of 
teachers  in  SAHE-grantee  activities  are  in  reform  types,  such  as  study  groups  and  mentoring.  In 
addition,  our  data  indicate  that  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program  are  substantially  longer  in  duration  than  activities  supported  under  the 
district  component  of  the  program.  Finally,  most  Eisenhower-assisted  activities  are  focused  on 
individual  teachers,  although  about  20  percent  of  teachers  in  district  activities  report  that  activities 
are  focused  on  whole  schools  or  departments. 

Our  data  can  be  used  to  provide  evidence  on  one  of  the 
Department  of  Education’s  Performance  Indicators  established 
for  the  Eisenhower  program.  Indicator  3.2  focuses  on  the  extent 
to  which  professional  development  supported  by  the  program  is 
sustained  throughout  the  school  year  (see  box).  As  shown  in 
Exhibit  3.3,  20  percent  of  teachers  participating  in  district 
activities  are  in  activities  that  extend  at  least  six  months;  and  46 
percent  of  teachers  participating  in  SAHE-grantee  activities  are 
in  activities  that  extend  beyond  six  months.  If  we  use  “at  least 
six  months”  as  the  standard  against  which  activities  are  to  be 
assessed,  then  district  activities  do  not  yet  meet  the  35  percent 
standard  set  in  the  Indicator,  while  SAHE-grantee  activities  exceed 
amount.19 

In  summary,  then,  on  average,  SAHE-grantee  activities  appear  to  share  more  of  the  structural 
features  associated  with  high-quality  professional  development  than  do  district  activities.  District 
activities  on  the  average  are  shorter,  but  the  average  duration  has  increased  substantially  since  the 
last  evaluation  was  conducted.  In  the  following  section,  we  turn  to  the  core  features  of  the 
experiences  that  take  place  during  Eisenhower-assisted  activities. 


Indicator  3.2  Sustained 
Professional  Development. 

By  1998,  35  percent  of  teachers 
participating  in  district-level 
Eisenhower-assisted  activities 
will  participate  in  activities  that 
are  a component  of  professional 
development  that  extends  over 
the  school  year;  by  2000,  over 
50  percent  will. 


the  standard  by  a substantial 


18  One  exception  to  this  general  pattern  may  be  found  in  the  recently  established  NSF  Local  Systemic  Change 
program,  which  incorporates  a requirement  that  all  elementary  teachers  in  the  target  area  participate  in  at  least  100 
hours  of  professional  development,  and  all  secondary  teachers  participate  in  at  least  130  hours  (Weiss  et  al.,  1998). 

19  The  Indicator  requires  that  activities  “are  a component  of  professional  development  that  extends  over  the  school 
year.”  It  is  possible  that  some  short-term  Eisenhower  activities  are  linked  to  other  activities,  and  these  “sequences” 
of  activities  extend  over  the  school  year.  If  so,  the  percent  of  Eisenhower-assisted  activities  extending  more  than  six 
months  may  understate  the  percent  of  activities  that  “are  a component  of  professional  development  that  extends  over 
the  school  year.” 
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Core  Features 


Section  Findings 

♦ Many  teachers  participating  in  Eisenhower-assisted  activities,  especially  SAHE-grantee 
activities,  report  that  the  activities  place  a major  emphasis  on  deepening  their  content 
knowledge  in  mathematics  and  science. 

♦ Most  Eisenhower-assisted  activities  incorporate  some  form  of  active  learning — for 
example,  preparation  of  lesson  plans  and  discussion  of  classroom  implementation. 

Fewer  activities  involve  other  aspects  of  active  learning — for  example,  observation  of 
teaching  and  examination  of  student  work. 

♦ Eisenhower-assisted  activities  generally  encourage  a coherent  process  of  teacher 
learning.  Most  teachers  participating  in  Eisenhower-assisted  activities  report  that  the 
activities  are  consistent  with  their  goals  and  aligned  with  state  and  district  standards  and 
assessments. 

In  the  previous  section,  we  focused  on  three  features  of  the  structure  of  professional 
development  activities  that  establish  the  setting  in  which  professional  development  takes  place:  the 
type  of  activity,  its  duration,  and  opportunities  for  collective  participation.  In  this  section,  we  turn  to 
the  substance  of  the  core  experiences  provided  as  the  professional  development  activity  unfolds. 

The  reauthorizing  legislation  for  the  Eisenhower  program  emphasizes  the  intent  to  support 
activities  that  are  of  “high  quality,”  but  it  does  not  spell  out  the  particular  characteristics  that  high- 
quality  professional  development  should  have.  However,  some  elements  of  quality  are  enumerated 
in  the  rationale  and  supporting  statements  accompanying  the  legislation.  For  example,  in  referring  to 
the  literature  on  professional  development,  Section  2001(4)(A)  states  that: 

professional  development  must  be  focused  on  teaching  and  learning  in  order  to  improve 
the  opportunities  of  all  students  to  achieve  higher  standards. 

And  Section  2001(4)(B),  drawing  on  the  literature,  states  that: 

effective  professional  development  focuses  on  discipline-based  knowledge  and  effective 
subject-specific  pedagogical  skills,  involves  teams  of  teachers,  and,  where  appropriate, 
administrators  and  pupil  services  personnel,  in  a school  and,  through  professional 
networks  of  teachers,  and,  where  appropriate,  teacher  educators,  administrators,  pupil 
services  personnel,  and  parents,  is  interactive  and  collaborative,  motivates  by  its  intrinsic 
content  and  relationship  to  practice,  builds  on  experience  and  learning-by-doing,  and 
becomes  incorporated  into  the  everyday  life  of  the  school. 

In  addition,  the  explicitly  stated  purpose  of  the  law  is  to  ensure  that  teachers  have  access  to 
professional  development  that  “reflects  recent  research  on  teaching  and  learning”  and  “includes 
strong  academic  content  and  pedagogical  components”  (Section  2002(2)(B)(C)). 
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In  addition  to  the  law,  we  have  drawn  on  the  literature  to  specify  the  dimensions  of  quality. 

In  the  discussion  that  follows,  we  examine  three  features  in  particular  that  the  literature  suggests  may 
be  related  to  improvements  in  teacher  knowledge  and  skills,  teaching  practices,  and  student 
achievement.  The  first  dimension  concerns  the  extent  to  which  Eisenhower-assisted  activities  focus 
on  strong  academic  content;  the  second  concerns  the  extent  to  which  activities  include  opportunities 
for  teachers  to  become  actively  engaged  in  learning  new  skills;  and  the  third  concerns  the  extent  to 
which  Eisenhower-assisted  activities  form  part  of  a coherent,  meaningful  program  of  professional 
development  that  is  consistent  with  teachers’  goals  and  related  to  teachers’  work. 

While  these  three  dimensions  are  not  specifically  identified  in  the  legislation,  they  are 
broadly  consistent  with  the  purposes  of  the  law.  In  particular,  the  Congressional  findings  and 
supporting  statements  for  the  law  clearly  encourage  professional  development  that  focuses  on  content 
(“discipline-based  knowledge  and  effective  subject-specific  pedagogy”),  that  encourages  active 
learning  (“builds  on  experience  and  learning  by  doing”),  and  that  encourages  a coherent  program  of 
professional  development  (“becomes  incorporated  in  the  everyday  life  of  the  school”)  (Section 
2001  (4)(B)). 

In  the  following  sections,  we  consider  each  of  these  three  core  features  in  turn,  drawing  on 
our  survey  and  case-study  data  to  assess  the  extent  to  which  Eisenhower-assisted  activities  reflect 
these  central  features  of  quality  professional  development. 

Focusing  on  Content 

Although  there  is  a large  literature  on  professional  development,  surprisingly  little  attention 
has  been  given  to  the  composition  and  characteristics  of  professional  development  activities.  In 
particular,  little  research  has  been  conducted  on  the  relative  efficacy  of  professional  development 
activities  that  focus  on  different  types  of  knowledge,  skills,  and  teaching  practices. 

The  available  descriptive  research  suggests  that  the  content  covered  during  professional 
development  activities  varies  along  at  least  four  dimensions.  First,  activities  vary  in  the  relative 
emphasis  they  give  to  the  content  teachers  are  expected  to  teach  and  the  teaching  methods  teachers 
are  expected  to  employ.  Some  activities  are  intended  primarily  to  improve  teachers’  knowledge  of 
subject-matter  content;  some  are  designed  to  improve  general  pedagogy  or  teaching  practices,  such 
as  classroom  management,  lesson  planning,  or  grouping  methods;  and  some  are  intended  to  improve 
what  Shulman  (1987)  has  termed  “pedagogical  content  knowledge”  - teaching  practices  in  specific 
content  domains,  such  as  the  teaching  of  multi-digit  addition  in  elementary  mathematics  or  forces 
and  motion  in  physics. 

Activities  also  vary  in  the  specificity  of  the  changes  in  teaching  practice  that  are  encouraged. 
Some  activities  focus  on  helping  teachers  use  particular  curriculum  materials  (e.g.,  new  textbooks, 
science  kits,  or  curriculum  replacement  units)  or  prescribed  teaching  strategies  (e.g.,  specific  student 
questioning  strategies).  Others  focus  on  general  principles,  giving  less  attention  to  specific  curricula 
or  strategies.  (See  Kennedy,  1998,  for  a discussion  of  this  distinction.) 


20  Not  all  professional  development  is  focused  on  knowledge  and  skills.  Some  activities  are  designed  to  increase 
teachers’  awareness  of  new  practices  rather  than  to  increase  knowledge  and  skills;  others  are  designed  to  build  or 
renew  teachers’  motivation  and  commitment  to  teaching,  without  necessarily  changing  teaching  practices. 


erIc 


3-20 


124 


In  addition,  activities  vary  in  the  goals  for  student  learning  emphasized.  Some  activities 
emphasize  helping  teachers  improve  student  performance  in  the  basic  skills — for  example, 
memorizing  facts  and  mastering  procedural  skills,  such  as  long  division  or  solving  linear  equations  in 
one  unknown.  Other  activities  focus  on  helping  teachers  improve  students’  conceptual 
understanding — for  example,  the  ability  to  explain  the  reasons  behind  a solution  strategy.21 

Finally,  activities  vary  in  the  emphasis  they  give  to  the  ways  students  learn  particular  subject 
matter.  Some  activities  give  considerable  emphasis  to  improving  teachers’  understanding  of  how 
children  learn,  by  focusing,  for  example,  on  common  student  preconceptions,  misconceptions,  and 
solution  strategies  in  specific  subject  domains.  Other  activities  focus  primarily  on  new  curricula  or 
teaching  methods,  while  giving  little  attention  to  the  ways  students  learn. 

Although  there  is  little  evidence  on  the  relative  effectiveness  of  professional  development 
activities  that  focus  on  different  types  of  knowledge,  skills,  and  teaching  practices,  a small  literature 
has  begun  to  emerge  focusing  on  these  issues.  In  particular,  an  emerging  body  of  work  suggests  that 
professional  development  that  focuses  on  subject-matter  content  and  how  children  learn  it  may  be  an 
especially  important  element  in  changing  teaching  practice  (e.g.,  Ball  & Cohen,  in  press;  Corcoran, 
1995).  In  part,  researchers  base  this  argument  on  the  fact  that  many  teachers  lack  strong  content- 
specific  teaching  skills.  Reynolds,  for  example,  in  a review  of  the  knowledge  base  for  elementary 
school  teachers,  concluded  that  “beginning  teachers  have  surprisingly  few  content-specific 
pedagogical  understandings”  (1995,  p.  214).  And  Rhine  (1998),  in  a discussion  of  the  role  of 
research  in  teaching,  pointed  out  that  “[rjeform-minded  teachers  are  hungry  for  continuing  education 
that  provides  novel  ways  to  address  content”  (p.  27). 

A number  of  authors  argue  that  professional  development  requires  a dual  focus  on  both 
knowledge  of  subject  matter  content  and  an  understanding  of  how  children  learn  specific  content. 
Hiebert  et  al.  (1996),  for  example,  argue  that  teaching  for  understanding  in  mathematics  requires  two 
forms  of  knowledge: 

knowledge  of  the  subject  to  select  tasks  that  encourage  students  to  wrestle  with  key  ideas  and 

knowledge  of  students’  thinking  to  select  tasks  that  link  with  students’  experience  and  for 

which  students  can  see  the  relevance  of  the  ideas  and  skills  they  already  possess  (p.  16). 

This  point  of  view  is  bolstered  by  several  recent  studies  of  the  effects  of  professional 
development  on  student  achievement.  Cohen  and  Hill  (1998)  conducted  a study  of  mathematics 
teaching  in  California,  based  on  data  on  teachers’  professional  development  experiences  and  school- 
level  data  on  student  performance  on  a mathematics  test  administered  statewide.  They  found  that, 
controlling  for  the  characteristics  of  students  enrolled,  average  mathematics  achievement  was  higher 
in  schools  in  which  teachers  had  participated  in  extensive  professional  development  focusing  on  the 
teaching  of  specific  mathematics  content  than  in  schools  in  which  teachers  had  not.  Participation  in 
professional  development  focusing  on  general  pedagogy,  however,  was  not  related  to  student 
achievement. 

Kennedy  (1998)  found  similar  results  in  a review  of  well-designed  experimental  studies  of 
the  relationship  between  professional  development  and  student  achievement  in  mathematics  and 


21  See  Chapter  2 for  a discussion  of  these  performance  goals. 
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science,  commissioned  as  part  of  this  evaluation.  The  review  found  that,  compared  to  more  general 
professional  development,  professional  development  that  focuses  on  specific  content  and  how 
students  learn  has  larger  positive  effects  on  student  achievement  outcomes,  especially  achievement  in 
conceptual  understanding. 

Based  on  this  emerging  evidence,  we  view  the  degree  of  content  focus  as  a central  dimension 
of  high-quality  professional  development.  To  examine  the  content  focus  of  Eisenhower-assisted 
activities,  we  asked  each  teacher  in  our  national  sample  to  indicate  the  degree  of  emphasis  the 
activity  in  which  the  teacher  participated  gave  to  deepening  content  knowledge  in  mathematics  and 
science,  using  a three-point  scale  (no  emphasis=zero,  minor  emphasis=one,  major  emphasis=two).22 

The  results  are  reported  in  Exhibit  3.5.  As  can  be  seen,  many  teachers  in  both  district  and 
SAHE-grantee  activities  report  a major  emphasis  on  content  in  the  activities  in  which  they 
participated,  although  teachers  in  district  activities  report  somewhat  less  focus  on  content  than 
teachers  in  SAHE-grantee  activities.  Overall,  51  percent  of  teachers  in  district  activities  report  a 
major  emphasis  on  content  and  35  percent  report  a minor  emphasis.  For  SAHE-grantee  activities,  68 
percent  of  teachers  report  a major  emphasis  and  28  percent  report  a minor  emphasis.  There  is  no 
significant  difference  in  content  focus  between  traditional  and  reform  activities. 


EXHIBIT  3.5 


Percent  of  Teachers  Reporting  a Major  Emphasis  on  Mathematics  and  Science 
Content  Knowledge  in  Eisenhower-assisted  Professional  Development  Activities 

(District  n=754,  SAHE  Grantee  n=243) 


68%  66,%  72% 


28%  29%  24% 


5°/o  5°/o  4°/o 


Overall  Traditional  Reform 

SAHE  Grantees 


Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  51  percent  of  the  teachers  who  participated  in  district  activities  report  that  a major 
emphasis  was  put  on  content  knowledge  in  their  activity.  Each  dot  represents  one  teacher.  As  the  number  of  teachers  at  one  data  point  (or 
value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  teachers  for  that 
particular  category.  The  number  on  the  top  of  the  distribution  is  the  percent  of  teachers  responding  to  the  category. 

Note.  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


22  We  did  not  ask  teachers  about  other  aspects  of  content-focus — for  example,  the  extent  to  which  the  activity 
emphasized  how  students  learn  specific  content  or  the  extent  to  which  it  focused  on  methods  of  teaching  specific 
content.  Items  on  these  aspects  of  content  focus  are  included  in  the  third  wave  of  the  teacher  longitudinal  survey, 
for  which  results  will  be  available  in  the  fall  of  1999. 


3-22 


126 


Although  our  survey  data  do  not  tell  us  whether  the  content  emphasized  is  of  high  quality, 
our  data  do  indicate  that  a substantial  proportion  of  Eisenhower-assisted  activities  are  tied  to  subject- 
matter  content.  Given  the  growing  literature  and  recent  research  emphasizing  the  potential 
relationship  between  content  focus  and  student  achievement,  these  results  are  encouraging. 

Promoting  Active  Learning 

A second  core  feature  of  professional  development  concerns  the  opportunities  provided  for 
teachers  to  become  actively  engaged  in  meaningful  discussion,  planning,  and  practice  as  part  of  the 
professional  development  activity.  (See,  for  example,  Lieberman,  1996;  and  Loucks-Horsley, 
Hewson,  Love,  & Stiles,  1998.)  Opportunities  for  active  learning  can  take  a number  of  forms, 
including  the  opportunity  to  observe  expert  teachers  and  to  be  observed  teaching;  the  opportunity  to 
plan  how  new  curriculum  materials  and  new  teaching  methods  will  be  used  in  the  classroom;  the 
opportunity  to  review  student  work  in  the  topic  areas  being  covered;  and  the  opportunity  to  lead 
discussions  and  engage  in  written  work. 

Lieberman  (1996)  argues  that,  for  teachers  to  change  their  teaching,  they  must  have  the 
opportunity  to  “talk,  think,  try  out,  and  hone  new  practices”  (p.  189).  Darling-Hammond  (1997b)  has 
characterized  professional  development  with  characteristics  such  as  “rich  professional  development.” 
According  to  Darling-Hammond,  rich  professional  development: 

is  centered  around  the  critical  activities  of  teaching  and  learning — planning  lessons, 
evaluating  student  work,  developing  curriculum — rather  than  around  abstractions  and 
generalities;  it  grows  from  investigations  of  practice  through  cases,  questions,  analysis, 
and  criticism;  and  it  is  built  on  substantial  professional  discourse  that  fosters  analysis  and 
communication  about  practice  and  values  in  ways  that  build  collegiality  and  standards  of 
practice  [emphasis  in  original]  (p.  323). 

The  importance  of  engaging  teachers  in  concrete  tasks  as  a part  of  professional  development 
is  given  some  support  in  a recent  study  of  34  exemplary  professional  development  activities  in 
mathematics  and  science,  supported  by  the  National  Science  Foundation,  the  Department  of 
Education,  and  other  federal  agencies  (Carey  & Frechtling,  1997).  As  part  of  an  evaluation  of  these 
activities,  teachers  were  surveyed  about  the  degree  to  which  the  activities  included  the  opportunity  to 
plan  how  the  concepts  and  materials  would  be  used  in  the  classroom,  as  well  as  the  opportunity  to 
engage  in  hands-on  activities,  challenging  problem  solving,  and  the  development  of  curriculum  units. 
Carey  and  Frechtling  indicate  that  teachers  who  participated  in  activities  that  offered  these 
opportunities  reported  more  change  in  knowledge,  skills,  and  teaching  practices  than  teachers  in 
activities  that  did  not  afford  these  opportunities. 

In  the  following  paragraphs,  we  examine  the  degree  to  which  opportunities  of  this  kind  are 
provided  in  our  national  sample  of  Eisenhower-assisted  activities.  Although  we  do  not  have 
information  on  the  extent  to  which  a particular  strategy  was  used  in  an  activity  nor  whether  it  was 
used  in  conjunction  with  conceptual,  in-depth  learning,  we  do  have  information  on  the  prevalence  of 
several  types  of  learning  strategies.  We  focus  in  particular  on  four  dimensions  of  active  learning: 
observing  and  being  observed  teaching;  planning  for  classroom  implementation;  reviewing  student 
work;  and  presenting,  leading,  and  writing. 
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Observing  and  Being  Observed 

One  element  of  active  learning  is  the  opportunity  for  teachers  to  observe  expert  teachers  and 
be  observed  teaching  in  their  own  classroom  and  obtain  feedback.  These  opportunities  can  take  a 
variety  of  forms.  Videotaped  lessons,  for  example,  can  offer  participants  an  opportunity  to  observe 
lesson  content,  teaching  strategies,  and  student  learning.  Teachers  can  visit  each  others’  classrooms 
to  observe  lessons.  Activity  leaders,  lead  teachers,  mentors,  and  coaches  can  observe  classroom 
teachers  and  engage  in  reflective  discussions  about  the  goals  of  a lesson,  the  tasks  employed, 
teaching  strategies,  and  student  learning. 

Eisenhower  funds  were  used  to  support  in-depth  opportunities  for  observation  in  West  City, 
Connecticut,  one  of  our  exploratory  case  sites.  As  part  of  the  portfolio  of  activities  offered  with 
Eisenhower  support  in  West  City,  two  skilled  in-house  facilitators  provide  coaching  and  support  for 
elementary  teachers,  including  demonstration  lessons  and  help  with  planning.  Teachers  who  receive 
intensive  coaching  generally  spend  one  to  two  hours  per  week  in  pre-lesson  discussion, 
demonstration  lessons,  and  post-lesson  feedback. 

To  determine  the  frequency  with  which  opportunities  of  this  kind  are  provided  as  part  of 
Eisenhower-assisted  activities,  we  asked  each  teacher  in  our  national  sample  how  the  activity  helped 
the  teacher  use  new  skills  in  the  classroom.  In  particular,  we  asked  each  teacher  whether  the  teacher 
received  coaching  or  mentoring  in  the  classroom  as  part  of  the  Eisenhower-assisted  activity;  whether 
the  teacher’s  teaching  was  observed  by  the  activity  leader(s)  and  feedback  was  provided;  and 
whether  the  teacher’s  teaching  was  observed  by  other  participants  and  feedback  was  provided.  We 
also  asked  whether  the  activity  was  evaluated  in  part  based  on  an  observation  of  the  teacher’s 
classroom. 

The  results,  shown  in  Exhibit  3.6,  indicate  that  relatively  few  teachers  report  opportunities  to 
observe  and  be  observed  as  part  of  Eisenhower-assisted  activities,  but,  overall,  S AHE-grantee 
activities  provide  more  opportunities  for  classroom  observation  than  do  district  activities.  For 
example,  22  percent  of  teachers  in  SAHE-grantee  activities  report  being  coached  in  the  classroom  as 
part  of  the  activities  in  which  they  participated,  compared  with  10  percent  of  teachers  in  district 
activities.  Similar  differences  appear  in  the  percentage  of  teachers  who  report  being  observed  by  the 
leader  (16  versus  five  percent),  who  report  being  observed  by  other  teachers  (15  versus  10  percent), 
and  who  report  that  the  activity  was  evaluated  in  part  based  on  observation  (22  versus  five  percent). 
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EXHIBIT  3.6 

Percent  of  Teachers  Reporting  That  They  Had  Opportunities  to  Observe  or  Be 
Observed  Teaching  in  Eisenhower-assisted  Professional  Development  Activities 

(District  n=783,  SAHE  Grantee  n=244) 


teaching  teaching  observed 

Types  of  opportunities  to  observe  and  be  observed  teaching 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  10  percent  of  the  teachers  who  participated  in  district  activities  report  that  they  had  an 
opportunity  to  receive  coaching  or  mentoring  in  the  classroom.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for 
each  category. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


Planning  Classroom  Implementation 

A second  element  of  active  learning  involves  the  opportunity  to  link  the  ideas  introduced 
during  professional  development  experiences  to  the  teaching  context  in  which  teachers  work.  The 
introduction  of  new  approaches  may  have  different  implications  depending  on  the  curriculum  in 
place  in  a teacher’s  school,  the  specific  textbooks  adopted  in  the  teachers’  classrooms,  and  the 
required  assessments  in  the  teachers’  districts,  as  well  as  the  characteristics  of  the  students  enrolled  in 
the  teachers’  classrooms,  including  the  material  covered  in  previous  grades  and  students’ 
expectations  for  classroom  instruction. 

To  assess  the  extent  to  which  Eisenhower  activities  provided  teachers  with  opportunities  to 
plan  classroom  implementation,  we  asked  each  teacher  in  our  national  sample  whether,  as  part  of  the 
activity  in  which  the  teacher  participated,  the  teacher  practiced  under  simulated  conditions,  with 
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feedback;  met  formally  with  other  activity  participants  to  discuss  classroom  implementation; 
communicated  with  the  leader(s)  of  the  activity  concerning  classroom  implementation;  met 
informally  with  other  participants  to  discuss  classroom  implementation;  and  developed  curricula  or 
lesson  plans  that  other  participants  or  the  activity  leader  reviewed. 

Exhibit  3.7  shows  that  opportunities  to  plan  classroom  implementation  are  a more  common 
part  of  Eisenhower-assisted  activities  than  opportunities  to  observe  and  be  observed  teaching.  But, 
as  in  our  results  for  observation,  SAHE-grantee  activities  are  more  likely  to  provide  opportunities  for 
planning  than  are  district  activities.  For  example,  32  percent  of  teachers  participating  in  district 
Eisenhower-assisted  activities  report  that  the  activities  in  which  they  participated  provided  the 
opportunity  to  discuss  classroom  implementation,  as  do  51  percent  of  teachers  in  SAHE-grantee 
activities.  Similarly,  36  percent  of  teachers  in  district  activities  report  that  the  activity  provided  an 
opportunity  to  communicate  with  the  leader  about  classroom  implementation,  as  do  54  percent  of 
teachers  in  IHE/NPO  activities. 


EXHIBIT  3.7 

Percent  of  Teachers  Reporting  Opportunities  to  Plan  Classroom  Implementation  in 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=783.  SAHE  Grantee  n=2441 


Conditions 

Types  of  opportunities  to  plan  classroom  implementation 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  29  percent  of  the  teachers  who  participated  in  district  activities  report  that  they  had  an 
opportunity  to  practice  under  simulated  conditions.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each 
category. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 
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Reviewing  Student  Work 


Another  element  of  active  learning  is  the  opportunity  to  examine  and  review  student  work. 

By  examining  students’  written  responses  to  problems,  for  example,  teachers  may  gain  an 
understanding  of  students’  preconceptions,  misconceptions,  and  solution  strategies  (Schifter,  1996; 
Carpenter  et  al.,  1989).  And,  by  observing  students’  responses  to  questions  posed  by  expert  teachers, 
teachers  may  gain  insight  into  students’  assumptions  and  reasoning.  Well-presented  examples  of 
student  work  can  provide  insight  into  the  types  of  difficulties  students  are  likely  to  have  with 
complex  subject  matter  and  the  materials  and  teaching  strategies  that  are  likely  to  be  effective.  In 
particular,  examples  of  student  work  may  help  teachers  develop  skills  in  diagnosing  student  problems 
and  designing  lessons  at  an  appropriate  level  of  difficulty.  Finally,  by  discussing  examples  of 
students’  work,  teachers  may  develop  shared  interpretations  of  the  kinds  of  problems  that  are  easy 
and  difficult  and  the  standards  of  performance  against  which  students  should  be  held. 

To  determine  the  extent  to  which  opportunities  to  review  student  work  are  provided  as  part  of 
Eisenhower-assisted  activities,  we  asked  each  teacher  in  our  national  sample  whether  the  teacher 
reviewed  student  work  or  scored  assessments  as  part  of  the  activity;  we  asked  whether  work 
completed  by  students  in  the  teacher’s  classroom  were  reviewed  by  other  activity  participants  or  the 
activity  leader;  and  we  asked  whether  student  outcomes  were  examined  as  part  of  an  evaluation  of 
the  activity. 

The  results,  shown  in  Exhibit  3.8,  indicate  that  opportunities  to  review  student  work  are 
relatively  rare,  although  they  are  somewhat  more  common  in  SAHE  grantee  than  district  activities. 
For  example,  19  percent  of  teachers  in  district  Eisenhower  activities  reported  that  the  activities 
provided  an  opportunity  to  review  student  work,  as  did  26  percent  of  teachers  in  SAHE-grantee 
activities;  and  nine  percent  of  teachers  in  district  activities  scored  assessments,  as  did  13  percent  of 
teachers  in  IHE/NPO  activities. 

Presenting,  Leading,  and  Writing 

Apart  from  opportunities  to  observe  teaching,  plan  classroom  implementation,  and  review 
student  work,  professional  development  activities  may  also  offer  teachers  the  opportunity  to  give 
presentations,  lead  discussions,  and  produce  written  work.  Active  participation  of  this  kind  may 
improve  outcomes  by  permitting  teachers  to  delve  more  deeply  into  the  substantive  issues 


To  examine  the  degree  to  which  these  forms  of  active  participation  are  provided  as  part  of 
Eisenhower-assisted  activities,  we  asked  each  teacher  in  our  national  sample  whether,  as  part  of  the 
activity,  the  teacher  gave  a lecture  or  presentation;  conducted  a demonstration  of  a lesson,  unit,  or 
skill;  led  a whole-group  discussion;  led  a small  group  discussion;  or  wrote  a paper,  report,  or  plan. 

The  results,  shown  in  Exhibit  3.9,  indicate  that  three  of  these  forms  of  active  learning — 
giving  a lecture  or  presentation,  conducting  a demonstration,  and  writing  a report — are  relatively 
common,  at  least  among  SAHE-grantee  activities.  Overall,  42  percent  of  teachers  in  SAHE-grantee 
activities  report  giving  a lecture  or  presentation  as  part  of  the  activity  in  which  they  participated,  43 
percent  conducted  a demonstration,  and  44  percent  wrote  a paper  or  report.  Fewer  teachers  who 
participated  in  district  activities  report  these  kinds  of  opportunities:  1 8 percent  of  teachers  in  district 
Eisenhower-assisted  activities  report  giving  a lecture  or  presentation,  24  report  conducting  a 
demonstration,  and  15  percent  report  writing  a paper. 


introduced. 
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Relatively  few  teachers  in  either  district  or  SAHE-grantee  activities  led  whole-group  or 
small-group  discussions,  although  discussion  is  often  emphasized  as  a strategy  to  be  encouraged  in 
the  teaching  of  mathematics  and  science. 


EXHIBIT  3.8 


Percent  of  Teachers  Reporting  Opportunities  to  Examine  Student  Work  in 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 
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■ Districts 
□ SAHE  Grantees 


Types  of  opportunities  to  examine  student  work 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit.  The  first  bar  shows  that  19  percent  of  the  teachers  who  participated  in  district  activities  report  that  they  had  an 
opportunity  to  review  student  work.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each  category. 

Note  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 
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EXHIBIT  3.9 

Percent  of  Teachers  Reporting  Opportunities  to  Present,  Lead,  and  Write  in 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 


■ Districts 
□ SAHE  Grantees 


Types  of  opportunties  to  present,  lead,  or  write 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  1 8 percent  of  the  teachers  who  participated  in  district  activities  report  that  they  had  an 
opportunity  to  give  a lecture  or  presentation.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each  category. 
Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


Overall  Index  of  Active  Learning 

In  the  previous  sections,  we  have  focused  on  the  extent  to  which  Eisenhower-assisted 
activities  provide  specific  types  of  opportunities — for  example,  the  opportunity  to  be  observed 
teaching  by  the  activity  leader,  score  student  assessments,  develop  a lesson  plan,  or  write  a report. 
The  description  we  have  provided  of  the  frequency  with  which  each  of  these  types  of  opportunities  is 
offered  provides  one  way  of  assessing  the  degree  to  which  Eisenhower-assisted  activities  support 
active  learning. 

A complementary  approach  to  examining  active  learning  is  to  ask  how  many  different  types 
of  active  learning  opportunities  Eisenhower-assisted  activities  generally  provide.  While  it  is  not 
clear  that  a particular  number  or  combination  of  active  learning  strategies  is  necessarily  more 
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effective,  activities  that  support  multiple  forms  of  active  learning  may  be  able  to  integrate  these 
opportunities  to  provide  strong  supports  for  teacher  change. 

An  Eisenhower-assisted  activity  provided  by  a SAHE-sponsored  IHE  in  Texas,  one  of  our  in- 
depth  case  states,  illustrates  the  ways  a varied  set  of  approaches  for  active  learning  can  build  upon 
one  another  to  facilitate  teacher  development.  Nineteen  teachers  participated  in  the  6lh-Grade 
Physical  Science  Enhancement  and  Math  Integration  program  offered  by  the  IHE.  The  program  was 
designed  to  create  a support  system  to  help  teachers  implement  teaching  practices  consistent  with  the 
national  standards  in  mathematics  and  science  and  the  Texas  Essential  Knowledge  and  Skills. 

During  a six-week  summer  institute,  teachers  kept  journals,  model  classrooms  were  used  to  help 
teachers  learn  new  strategies,  and  instructors  videotaped  teachers  in  the  classroom  and  provided 
feedback.  In  addition,  the  project  encouraged  participants  to  form  a network  to  provide  support 
during  the  school  year.  Finally,  the  activity  leaders  spent  about  200  hours  in  the  teachers’  classrooms 
during  the  following  school  year,  observing  and  providing  follow-up  and  guidance. 

To  examine  the  number  of  types  of  active  learning  opportunities  of  this  kind  provided  in  the 
activities  in  which  our  national  sample  of  teachers  participated,  we  created  a composite  index, 
summing  all  of  the  types  shown  in  Exhibits  3.6,  3.7,  3.8,  and  3.9.  Because  our  survey  included  four 
items  to  measure  opportunities  for  observation,  five  for  planning,  four  for  reviewing  student  work, 
and  five  for  presenting/writing,  simply  summing  the  18  types  of  opportunities  included  would  give 
more  weight  to  planning  and  presenting/writing  than  to  observing  and  reviewing  student  work.  Thus, 
in  computing  the  index,  we  weighted  each  of  the  four  items  pertaining  to  observation  and  the  four 
items  pertaining  to  student  work  by  1 .25.  This  produces  an  index  that  runs  from  zero  (no 
opportunities  were  provided  for  active  learning)  to  20  (all  types  of  active  learning  were  provided). 

The  results,  shown  in  Exhibit  3.10,  indicate  that  activities  vary  widely  in  the  number  of 
opportunities  they  provide  for  active  learning,  but,  overall,  SAHE-grantee  activities  provide  more 
opportunities  for  active  learning  than  district  activities.  On  the  average,  teachers  report  that  district 
activities  provide  3.6  types  of  active  learning  (of  the  maximum  of  20  possible  on  the  scale),  but  the 
range  is  enormous.  About  21  percent  of  teachers  are  in  district  activities  that  provide  no 
opportunities  for  active  learning,  while  some  are  in  activities  providing  more  than  eight  types.  On 
the  average,  teachers  in  SAHE-grantee  activities  report  5.8  types  of  active  learning,  although,  as  for 
districts,  there  is  a wide  range.  About  nine  percent  of  teachers  in  SAHE-grantee  activities  report  no 
opportunities  for  active  learning. 

As  anticipated,  reform  activities  tend  to  offer  more  opportunities  for  active  learning  than 
traditional  activities,  for  activities  supported  through  the  district  component  of  the  program.  On 
average,  teachers  participating  in  district  reform  activities  report  4.4  types  of  active  learning, 
compared  with  3.4  types  for  traditional  activities.  There  is  little  difference  in  opportunities  for  active 
learning  between  traditional  and  reform  activities  supported  through  the  SAHE  component  of  the 
program. 

Our  data  on  active  learning  opportunities  suggest  that  some  Eisenhower-assisted  activities — 
particularly  those  supported  in  the  SAHE  component  of  the  program — provide  a diverse  set  of  active 
learning  activities.  But  many  Eisenhower-assisted  activities — especially  traditional  activities 
supported  under  the  district  component  of  the  program — do  not  yet  incorporate  the  kinds  of  active 
learning  opportunities  the  literature  suggests  are  required  to  support  ongoing  improvement  in  teacher 
knowledge,  skills,  and  teaching  practice.  Thus,  this  is  an  area  in  which  further  program  improvement 
efforts  might  focus. 
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EXHIBIT  3.10 


Number  of  Opportunities  for  Active  Learning  in  Eisenhower-assisted  Professional 
Development  Activities,  as  Reported  by  Teachers 
(District  n=767,  SAHE  Grantee  n=244) 


7 5.8  7 5.9  4 5 7 


Overall  Traditional  Reform 

SAHE  Grantees 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  those  teachers  who  participated  in  district  activities  reported  that  they 
were  given  3.6  opportunities  for  active  learning,  where  0 indicates  no  opportunity  and  20  indicates  full  opportunities  on  all  18  different  active 
learning  items.  Traditional  vs.  reform  types  of  activities  differed  significantly  in  the  opportunities  for  active  learning  for  the  district 
component  activities,  but  not  for  the  SAHE-grantee  component  activities.  Each  dot  represents  one  teacher.  As  the  number  of  teachers  at  one 
data  value  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  teachers  for  that 
particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower- assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


Nurturing  Coherence 


The  third  core  feature  of  professional  development  concerns  the  extent  to  which  professional 
development  activities  are  perceived  by  teachers  to  be  a part  of  a coherent  program  of  teacher 
learning.  Up  until  this  point,  in  examining  the  quality  of  professional  development  activities 
supported  by  the  Eisenhower  program,  we  have  focused  on  experiences  that  occur  as  part  of  the 
activity  itself — the  content  covered  and  the  opportunities  for  active  learning  provided.  To  some 
extent,  however,  the  quality  of  a professional  development  experience  is  also  a function  of  the 
relationship  of  the  experience  itself  to  other  aspects  of  a teacher’s  work  (Darling-Hammond  & 
McLaughlin,  1996).  Developing  coherent  knowledge-based  visions  are  important  conditions  for 
learning  (McCarthey  & Peterson,  1993).  Professional  development  for  teachers  is  frequently 
criticized  on  the  ground  that  the  activities  are  disconnected  from  one-another — in  other  words, 
individual  activities  do  not  form  part  of  a coherent  program  of  teacher  learning  and  development.  A 
professional  development  activity  is  more  likely  to  be  effective  in  improving  teachers’  knowledge 
and  skills  if  it  forms  a coherent  part  of  a wider  set  of  opportunities  for  teacher  learning  and 
development. 

The  coherence  of  a teacher’s  professional  development  experience  can  be  assessed  in  at  least 
three  ways.  First,  coherence  can  be  judged  in  terms  of  the  degree  to  which  an  individual  professional 
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development  activity  builds  on  what  teachers  have  already  learned  and  is  followed  up  with  other, 
more  advanced  work.  The  coherence  of  an  activity  can  also  be  judged  in  terms  of  the  extent  to  which 
the  activity  emphasizes  content  and  pedagogy  aligned  with  national,  state,  and  local  standards, 
frameworks,  and  assessments.  Finally,  the  coherence  of  an  activity  can  be  judged  on  the  basis  of  the 
extent  to  which  the  activity  supports  teachers  in  developing  sustained,  ongoing  professional 
communication  with  other  teachers  who  are  trying  to  change  their  teaching  in  similar  ways. 

In  the  following  sections,  we  consider  each  of  these  aspects  of  coherence  in  turn. 

Connections  with  Goals  and  Other  Activities 

One  way  to  assess  whether  a professional  development  activity  is  part  of  a coherent  program 
of  teacher  learning  is  to  ask  whether  the  activity  builds  on  earlier  activities  and  is  followed  up  with 
later,  more  advanced  work.  Are  the  individual  professional  development  opportunities  provided  for 
teachers  linked  so  they  build  upon  one  another?  Do  professional  development  activities  make  it 
possible  for  teachers  to  advance  from  more  basic  to  more  advanced  topics  and  skills,  covered  in 
greater  depth?  Or,  do  professional  development  opportunities  tend  to  repeat  content  from  earlier 
activities,  at  the  same  depth?23 

Furthermore,  are  the  opportunities  provided  linked  to  teachers’  goals  for  development?  Do 
teachers  perceive  that  the  available  activities  are  central  to  the  issues  of  teaching  and  learning  that 
they  believe  need  more  attention? 

To  address  these  issues,  we  asked  each  teacher  in  our  national  sample  to  report  the  extent  to 
which  the  activity  the  teacher  attended  was  consistent  with  the  teacher’s  goals  for  professional 
development;  based  explicitly  on  what  the  teacher  had  learned  in  earlier  professional  development 
experiences;  and  followed  up  with  activities  that  built  upon  what  was  learned  in  this  professional 
development  activity.  Teachers  responded  on  a scale  from  one  to  five,  where  one=not  at  all  and 
five=to  a great  extent. 

The  results,  shown  in  Exhibit  3.1 1,  indicate  that  almost  all  teachers  believe  the  Eisenhower- 
assisted  activities  in  which  they  have  participated  are  consistent  with  their  goals:  79  percent  of 
teachers  in  district  activities  report  that  the  activities  in  which  they  participated  were  quite  consistent 
with  their  goals  (that  is,  they  reported  a value  of  four  or  five  on  the  five-point  response  scale);  and  88 
percent  of  teachers  in  SAHE-grantee  activities  reported  that  the  activities  were  consistent  with  their 
goals. 


Many  fewer  teachers  report  that  the  Eisenhower-assisted  activities  build  upon  earlier 
activities:  35  percent  of  teachers  in  district  activities  and  39  percent  of  teachers  in  SAHE-grantee 
activities  report  that  the  activity  built  on  earlier  activities. 

Somewhat  more  teachers  report  that  the  Eisenhower-assisted  activities  in  which  they 
participated  wer e followed  up  with  additional  activities  than  reported  that  Eisenhower-assisted 
activities  built  upon  earlier  work.  About  53  percent  of  teachers  in  district  activities  report  that  they 
were  followed  up  with  more  advance  work,  as  did  70  percent  of  teachers  in  SAHE-grantee  activities. 


23  To  some  extent,  the  issues  here  are  parallel  to  those  posed  in  assessing  the  learning  opportunities  provided  for 
students. 


EXHIBIT  3.11 


Percent  of  Teachers  Reporting  That  Eisenhower-assisted  Professional  Development 
Activities  are  Related  to  Their  Other  Professional  Development  Experiences  (District 

n=748  to  760,  SAHE  Grantee  n=238  to  239) 


Activities 

Relationship  to  other  experiences 


Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  79  percent  of  the  teachers  who  participated  in  district  activities  report  that  the  activity  was 
consistent  with  their  own  goals.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each  category. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program.  Due  to  missing  data,  the  district  n ranges  from  748  to  760  and  the  SAHE  Grantee  n ranges  from  238  to  239. 


Alignment  with  State  and  District  Standards  and  Assessments 

A second  aspect  of  coherence  concerns  the  alignment  of  the  content  and  pedagogy 
emphasized  in  the  activities  with  national,  state,  and  local  frameworks,  standards,  and  assessments. 
Teachers  receive  guidance  about  what  to  teach  and  how  to  teach  it  from  multiple  sources,  including 
the  material  covered  in  formal  professional  development,  as  well  as  in  their  pre-service  education, 
textbooks,  national  standards,  state  and  local  policies  and  assessments,  and  the  professional  literature 
(Cohen  & Spillane,  1992).  To  the  extent  that  these  sources  provide  a coherent  set  of  goals  for 
teaching  and  learning,  they  may  facilitate  teachers’  efforts  to  improve  their  teaching  practices.  To 
the  extent  that  they  conflict,  however,  the  sources  of  guidance  may  create  tensions  that  impede 
teachers’  efforts  to  develop  their  teaching  in  a consistent  direction  (Grant,  Peterson,  & Shojgreen- 
Downer,  1996). 
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Efforts  to  align  professional  development  with  state  and  district  frameworks,  standards,  and 
assessments  offer  one  approach  to  increasing  the  coherence  of  the  instructional  guidance  teachers 
receive.  The  process  of  aligning  professional  development  with  state  and  district  standards  and  other 
policies  can  take  a number  of  forms.  For  example,  professional  development  activities  can  be  chosen 
to  reflect  the  topics  emphasized  in  state  and  district  standards.  Or,  professional  development 
activities  can  focus  on  the  goals  for  student  learning  emphasized  in  state  assessments  or  the 
pedagogical  methods  emphasized  in  state  curriculum  frameworks  (Webb,  1998). 

A professional  development  activity  in  Riverside,  Washington,  one  of  our  case-study 
districts,  illustrates  the  use  of  Eisenhower  funds  to  encourage  professional  development  that  is 
aligned  with  standards.  The  Riverside  Elementary  Science  Kit  project  is  an  ongoing  process  that 
started  in  1990,  when  teachers  began  attending  a series  of  extended  workshops  on  science  concepts. 
After  several  years  of  related  workshops,  building  from  simple  to  more  advanced  topics,  between  30 
and  50  teachers  were  prepared  and  ready  to  begin  creating  kits  They  began  creating  three  to  four  kits 
in  each  grade  level:  one  concept  kit,  and  two  or  three  content  kits  per  grade. 

Thirty-one  kits  have  now  been  completed,  focusing  on  earth,  life,  and  physical  science. 
Currently,  Eisenhower-assisted  workshops  are  being  used  to  develop  standards  for  the  scoring  of  the 
student  work  included  in  the  kits.  These  workshops  are  designed  to  help  match  state  and  national 
learning  standards  to  the  scoring  methods  used  for  the  kits.  After  completing  the  workshops,  the 
teachers  are  expected  to  know  which  state  and  national  standards  the  science  kits  satisfy,  and  they 
should  be  able  to  understand  the  goals  for  student  learning  embodied  in  each  kit. 

To  obtain  a measure  of  the  alignment  of  Eisenhower-supported  professional  development 
with  state  and  district  standards,  we  asked  each  teacher  in  our  national  sample  to  indicate  the  extent 
to  which  the  activity  in  which  the  teacher  participated  was  aligned  with  state  or  district  standards  and 
curriculum  frameworks,  and  with  state  and  district  assessments.  Teachers  were  asked  to  respond 
using  a five-point  scale,  from  l=not  aligned  at  all  to  5=aligned  to  a great  extent. 

The  results,  displayed  in  Exhibit  3.12,  show  that  most  teachers  report  that  the  activities  they 
participated  in  were  aligned  with  state  and  district  standards,  frameworks,  and  assessments,  although 
teachers  report  somewhat  more  alignment  between  professional  development  with  standards  and 
frameworks  than  assessments.  Overall,  80  percent  of  teachers  participating  in  Eisenhower-assisted 
activities  report  that  the  activities  were  well  aligned  with  state  and  district  standards  (reporting  a four 
or  five  on  the  five-point  scale),  while  68  percent  report  that  the  activities  were  well  aligned  with  state 
and  district  assessments.  The  results  for  SAHE-grantee  activities  are  nearly  identical.  Although  our 
data  do  not  permit  us  to  judge  the  depth  of  the  alignment  reported  in  our  surveys,  our  results  do 
indicate  that  teachers  perceive  that  the  activities  in  which  they  participate  are  quite  consistent  with 
state  and  district  standards  and  frameworks,  and  moderately  consistent  with  state  and  district 
assessments. 
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EXHIBIT  3.12 


Percent  of  Teachers  Reporting  That  Their  Eisenhower-assisted  Activities  are  Aligned 
with  State  and  District  Standards,  Frameworks,  and  Assessments  (District  n=748  to 

753,  SAHE  Grantee  n=238) 


District  Standards  District  Assessments 

Alignment  with  state  and  district  standards 
and  assessments 


Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1 998. 

How  to  read  this  exhibit : The  first  bar  shows  that  80  percent  of  the  teachers  who  participated  in  district  activities  report  that  the  activity  was 
designed  to  support  state  or  district  standards.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each  categoiy. 
Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program.  Due  to  missing  data,  the  district  n ranges  from  748  to  753;  the  SAHE  Grantee  n is  238. 


Communication  with  Others 

The  third  dimension  of  coherence  concerns  the  ways  in  which  professional  development 
activities  encourage  professional  communication  among  teachers  who  are  engaged  in  efforts  to 
reform  their  teaching  in  similar  ways.  Efforts  to  change  teaching  practices  generally  engender 
problems,  challenges,  and  dilemmas  as  teachers  attempt  to  carry  out  in  the  classroom  ideas  that  were 
introduced  in  professional  development  activities.  An  ongoing  discussion  among  teachers  who 
confront  similar  issues  can  facilitate  change  by  encouraging  the  sharing  of  solutions  to  problems,  as 
well  as  by  reinforcing  the  sense  that,  with  time,  improvement  is  possible.  There  is  some  evidence, 
for  example,  that  networks  of  teachers  involved  in  change  can  help  sustain  motivation  (Lieberman  & 
McLaughlin,  1992).  In  addition,  by  sharing  methods,  discussing  written  work,  and  reflecting  on 
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problems  and  solutions,  teachers  may  foster  a better  understanding  of  the  goals  for  student  learning 
that  proposed  changes  in  teaching  imply.  Communication  with  other  teachers  is  another  dimension 
of  coherence. 

To  measure  the  extent  to  which  teachers  in  our  national  sample  were  encouraged  to  establish 
professional  communication  as  part  of  the  Eisenhower-assisted  activities  in  which  they  participated, 
we  asked  the  teachers  whether  they  had  discussed  what  they  learned  with  other  teachers  in  their 
school  or  department  who  did  not  attend  the  activity;  whether  they  had  discussed  or  shared  what  they 
learned  with  administrators  (e.g.,  principal  or  department  chair);  and  whether  they  had 
communicated,  outside  of  formal  meetings  held  as  part  of  the  activity,  with  participants  in  the 
activity  who  teach  in  other  schools. 

The  results  are  shown  in  Exhibit  3.13.  The  data  indicate  that  most  teachers  report  discussing 
what  they  learned  with  other  teachers  in  their  school  who  did  not  attend  the  professional  development 
activity,  and  most  also  discuss  what  they  learned  with  school  administrators.  Somewhat  fewer 
teachers,  however,  report  communicating  with  teachers  in  other  schools. 

EXHIBIT  3.13 

Percent  of  Teachers  Reporting  That  They  Communicate  with  Other  Teachers  About 
Their  Experiences  in  Eisenhower-assisted  Professional  Development  Activities 

(District  n=783,  SAHE  Grantee  n=244) 


Teachers  Administrators  Teachers  in  Other  Schools 

Types  of  communication 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit.  The  first  bar  shows  that  73  percent  of  the  teachers  who  participated  in  district  activities  report  that  they  discussed  what 
they  learned  with  other  teachers  in  their  school  who  did  not  attend  the  activity.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of 
teachers  for  each  category. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower- assisted  activities  provided  through  the  district  component  of  the  program. 
“SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of  the  program. 


3-36 


Overall  Index  of  Coherence 


To  provide  a composite  measure  of  the  overall  extent  to  which  Eisenhower-assisted 
activities  are  a part  of  a coherent  program  of  professional  development,  we  combined  the  items  that 
comprise  our  three  specific  dimensions  of  coherence.  The  composite  sums  the  items  shown  in 
Exhibits  3.11,  concerning  connections  to  teachers’  goals  and  other  professional  development 
experiences;  Exhibit  3.12,  concerning  alignment;  and  Exhibit  3.13,  concerning  professional 
communication.  Because  three  items  are  available  for  the  first  and  third  of  these  dimensions,  while 
only  two  items  are  available  for  the  second,  we  weighted  the  items  for  the  second  dimension  by  1.5. 
This  produces  a scale  that  runs  from  zero  (the  activity  did  not  include  any  of  the  types  of  coherence 
we  measured)  to  nine  (the  activity  provided  all  of  the  forms  we  measured). 

The  results,  displayed  in  Exhibit  3.14,  indicate  that  both  district  and  SAHE-grantee 
activities  incorporate  many  of  the  features  of  coherence  we  measured,  although  the  variation 
across  activities  is  substantial.  The  average  number  of  types  of  coherence  reported  for  district 
activities  is  6.0  (on  our  0-9  scale),  and  the  average  for  SAHE-grantee  activities  is  6.4.  Reform 
activities  incorporate  somewhat  more  forms  of  coherence  than  traditional  activities  (6.3  versus 
5.9  for  districts,  and  7.0  versus  6.2  for  SAHE  grantees). 


EXHIBIT  3.14 

Degree  to  Which  Eisenhower-assisted  Professional  Development  Activities  Foster 
Coherence,  as  Reported  by  Teachers  (District  n=747,  SAHE  Grantee  n=235) 


I 

T 

▼ 

i 


± 


| 6.2 
1 
T 

T 

* 


l 

i 7.0 


Overall  Traditional  Reform 

SAHE  Grantees 


Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  those  teachers  who  participated  in  district  activities  reported  that  the 
activity  provides  a 6.0  coherence  scale  score,  where  zero  indicates  no  coherence  and  9.0  indicates  full  coherence  on  all  eight  different 
coherence  items.  Compared  to  the  traditional  types  of  activities,  reform  types  were  significantly  higher  in  terms  of  coherence.  Each  dot 
represents  one  teacher.  As  the  number  of  teachers  at  one  data  value  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each 
distribution  represents  the  distribution  of  teachers  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


These  results  indicate  that  most  teachers  perceive  Eisenhower-assisted  activities  to  be 
integrated  in  a variety  of  ways  into  their  professional  lives.  This  clearly  suggests  that  progress  has 
been  made  since  the  previous  Eisenhower  evaluation,  when  Eisenhower-assisted  activities  were 
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described  as  “one-shot”  (Knapp  et  al.,  1991).  Given  the  survey  data  we  have  available,  we  cannot  be 
sure  how  deeply  integrated  Eisenhower-assisted  activities  are  into  the  fabric  of  teachers’  professional 
work,  but  we  can  conclude  that  many  activities  are  fostering  important  linkages  with  teachers’  other 
professional  development,  state  and  district  standards,  and  other  teachers. 

In  addition,  as  we  had  anticipated,  reform  activities  are  more  likely  to  foster  coherence  than 
traditional  activities,  such  as  workshops  and  institutes.  Because  reform  types  of  activities,  such  as 
mentoring,  coaching,  and  study  groups,  frequently  take  place  during  the  regular  school  day,  it  is 
undoubtedly  easier  for  such  activities  to  foster  communication  among  teachers.  The  fact  that  reform 
activities  are  tied  to  the  daily  life  of  the  school  may  also  help  such  activities  foster  other  aspects  of 
coherence — particularly  alignment  with  standards. 

Summary:  Core  Features 

Our  results  on  the  core  features  of  Eisenhower-assisted  professional  development  activities 
provide  a mixed  picture.  We  examined  three  core  dimensions  of  the  professional  development 
activity:  content  focus,  active  learning,  and  coherence.  With  respect  to  the  first,  most  teachers 
participating  in  Eisenhower-assisted  activities  report  that  the  activities  in  which  they  participated 
placed  a major  emphasis  on  deepening  their  content  knowledge  in  mathematics  and  science.  This  is 
quite  encouraging,  given  the  conclusions  in  recent  literature  on  the  potential  benefits  of  professional 
development  focused  on  specific  subject-matter  content  (Kennedy,  1998;  Cohen  & Hill,  1998). 

With  respect  to  opportunities  for  active  learning,  the  results  appear  less  positive.  While  most 
teachers  report  that  the  Eisenhower-assisted  activities  in  which  they  participated  provide  an 
opportunity  to  plan  classroom  implementation  and  to  give  presentations  or  demonstrations,  relatively 
few  provide  an  opportunity  for  teachers  to  observe  or  be  observed  or  to  examine  student  work. 

There  is  a growing  body  of  research  to  suggest  that  these  forms  of  active  learning  are  critical 
components  of  effective  professional  development  (Schifter,  1996;  Carpenter  et  al.,  1989).  If  so, 
Eisenhower-assisted  activities  do  not  completely  meet  standards  of  high  quality  in  this  respect. 

Finally,  with  respect  to  coherence,  our  results  are  reasonably  positive.  Most  teachers  report 
that  the  activities  in  which  they  participated  were  connected  to  their  goals  for  professional 
development  and  other  professional  learning  opportunities,  aligned  with  state  and  district  standards 
and  assessments,  and  led  to  professional  communication  with  other  teachers. 

Our  results  also  support  two  other  conclusions.  First,  teachers  participating  in  activities 
supported  through  the  SAHE  component  of  the  program  are  more  likely  to  report  that  the  activities 
emphasized  mathematics  and  science  content,  have  opportunities  for  active  learning,  and  more 
aspects  of  coherence  than  teachers  participating  in  district  activities.  As  we  will  discuss  later  in  this 
chapter,  these  differences  can  be  explained  in  part  by  the  longer  duration  of  SAHE-grantee  activities. 

In  addition,  our  results  indicate  that  reform  activities  are  more  likely  than  traditional  activities 
to  provide  active  learning  opportunities  and  more  likely  to  encourage  coherence.  The  differences 
between  reform  and  traditional  types  of  professional  activities  in  opportunities  for  active  learning  are 
not  as  large  as  the  differences  between  SAHE-grantee  and  district  activities,  however,  and,  in  relation 
to  the  overall  variation  among  activities,  are  only  moderate  in  size.  The  reform-traditional 
differences  in  coherence  are  larger,  however,  suggesting  that  reform  activities  are  especially  valuable 
in  encouraging  integration  of  professional  development  with  other  aspects  of  teachers’  work. 
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Our  results  provide  evidence  that  can  be  used  to 
assess  the  Eisenhower  program  with  respect  to  two  of 
the  Department  of  Education’s  Performance  Indicators 
for  the  program  (see  box).  The  first  indicator, 
pertaining  to  alignment,  sets  as  a standard  that  over  50 
percent  of  teachers  participating  in  Eisenhower-assisted 
activities  will  participate  in  activities  that  are  aligned 
with  high  standards.  As  shown  in  Exhibit  3.12,  80 
percent  of  teachers  participating  in  Eisenhower-assisted 
activities  report  that  the  activities  were  designed  to 
support  state  and  district  standards — far  more  than  the 
standard  of  50  percent.  We  do  not  know  the  depth  of 
alignment  this  represents,  nor  whether  the  state  and 
district  standards  with  which  Eisenhower  activities  are 
aligned  are  truly  “high.”  But  these  results  are 
encouraging. 

The  second  indicator,  pertaining  to  quality,  sets  as  a standard  that  over  50  percent  of  teachers 
participating  in  Eisenhower-assisted  professional  development  will  participate  in  activities  that  are  of 
high  quality.  Here,  we  have  multiple  measures  that  can  be  used.  Our  data  on  content  focus,  the  first 
core  feature,  indicate  that  5 1 percent  of  teachers  participating  in  Eisenhower-assisted  activities 
participate  in  activities  that  place  a major  emphasis  on  content.  By  this  measure,  Eisenhower 
activities  meet  the  standard  set  in  the  indicator.  Our  data  on  the  active  learning,  the  second  core 
feature,  are  more  mixed;  fewer  than  50  percent  of  teachers  are  in  activities  providing  some  of  the  key 
dimensions  of  active  learning,  such  as  the  opportunity  to  observe  and  be  observed,  or  the  opportunity 
to  review  student  work  (see  Exhibits  3.7  and  3.8).  While  more  teachers  report  opportunities  to  plan 
classroom  implementation  and  to  conduct  presentations  and  demonstrations,  less  than  half  of  teachers 
in  district  activities  report  most  of  these  experiences.  Finally,  our  data  on  coherence,  the  third  core 
feature,  are  reasonably  positive.  More  than  50  percent  of  teachers  participating  in  district  activities 
report  that  the  activities  in  which  they  participated  are  consistent  with  their  goals  were  followed  up 
with  later  experiences,  were  aligned,  and  nurtured  professional  communication. 

In  the  next  section,  we  turn  our  attention  from  the  core  features  of  Eisenhower-assisted 
activities  to  their  reported  effects  on  teacher  knowledge,  skills,  and  teaching  practices. 


Indicator  2. 1 District-Level  Professional 
Development.  By  1998,  over  50  percent  of 
teachers  participating  in  district-level  or 
higher  education  Eisenhower-assisted 
professional  development  will  participate  in 
activities  that  are  aligned  with  high 
standards.  By  2000,  over  75  percent  will. 

Indicator  3.1  High  Quality.  By  1998, 
over  50  percent  of  teachers  participating  in 
district-level,  Eisenhower-assisted 
professional  development  activities  will 
participate  in  activities  reflecting  best 
practices,  including  a focus  on  continuous 
improvement.  By  2000,  over  75  percent 
will. 


Teacher  Outcomes 


Section  Findings 

♦ Many  teachers  report  that  participation  in  Eisenhower-assisted  activities  enhanced  their 
knowledge  and  skills.  For  example , almost  half  of  the  teachers  participating  in  district 
Eisenhower  activities  report  that  participation  enhanced  their  knowledge  of  mathematics 
and  science ; nearly  70  percent  of  teachers  participating  in  IHE/NPO  activities  report 
enhanced  knowledge  of  mathematics  and  science. 
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♦ Many  teachers  also  report  improvements  in  their  classroom  instruction  as  a consequence 
of  participation  in  Eisenhower-assisted  activities.  For  example,  55  percent  of  the 
teachers  participating  in  district  Eisenhower  activities  and  66  percent  of  teachers 
participating  in  IHE/NPO  activities  report  increasing  the  cognitive  challenge  of 
classroom  instruction. 

The  main  goal  of  the  Eisenhower  Professional  Development  Program  is  to  increase  teachers’ 
knowledge  and  skills  in  order  to  improve  their  teaching  practice,  which  in  turn  will  increase  student 
learning  and  achievement.  The  law  requires  that  states  develop  professional  development  plans 
designed  to  give  teachers  “the  knowledge  and  skills  necessary  to  provide  all  students  the  opportunity 
to  meet  challenging  State  content  standards  and  challenging  State  student  performance  standards” 
(Section  2205(B)(2)(b)).  Also  it  requires  that  LEAs  use  Eisenhower  funds  to  “give  teachers  and 
administrators  the  knowledge  and  skills  to  provide  students  with  the  opportunity  to  meet  challenging 
State  or  local  content  standards  and  student  performance  standards”  (Section  2210(b)(1)). 

In  the  following  paragraphs,  we  examine  the  extent  to  which  teachers  participating  in 
Eisenhower-assisted  activities  report  outcomes  consistent  with  these  legislative  purposes,  focusing 
first  on  the  extent  to  which  teachers’  knowledge  and  skills  are  enhanced  as  a results  of  participation, 
and  then  on  improvements  in  teaching  practice. 

Teacher  Knowledge  and  Skills 

To  assess  the  effects  of  participation  on  teachers’  knowledge  and  skills,  we  asked  each 
teacher  in  our  national  sample  to  indicate  the  degree  to  which  his  or  her  knowledge  and  skills  were 
enhanced  as  a result  of  participation  in  the  specific  Eisenhower-assisted  activity  that  drew  the  teacher 
into  the  sample.  We  asked  each  teacher  to  indicate  the  extent  to  which  knowledge  and  skills  had 
been  enhanced  in  each  of  the  following  areas: 

♦ Curriculum  (e.g.,  units,  texts,  standards) 

♦ Instructional  methods 

♦ Approaches  to  assessment 

♦ Use  of  technology  in  instruction  (e.g.,  computers,  graphing  calculators) 

♦ Strategies  for  teaching  diverse  student  populations  (e.g.,  students  with  disabilities,  from 
underrepresented  populations,  economically  disadvantaged,  limited  English  proficient, 
range  of  abilities) 

♦ Deepening  knowledge  of  mathematics 

Teachers  reported  their  responses  using  a five-point  scale,  where  l=not  at  all  and  5=to  a great 
extent.  Our  data,  shown  in  Exhibit  3.15,  indicate  that  teachers  who  participated  in  SAHE-grantee 
activities  tended  to  report  more  change  in  their  knowledge  and  skill  than  teachers  who  participated  in 
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district  activities.24  For  example,  68  percent  of  teachers  participating  in  SAHE-grantee  activities 
reported  that  their  participation  led  to  improvements  in  their  content  knowledge  in  mathematics  and 
science  (responding  with  a four  or  five  on  the  five-point  scale)  compared  to  48  percent  of  teachers  in 
district  activities.  Similarly,  64  percent  of  teachers  in  SAHE-grantee  activities  reported  enhanced 
knowledge  and  skills  in  curriculum,  and  79  percent  in  instructional  methods,  compared  with  56  and 
63  percent,  respectively,  for  district  activities.  In  addition,  teachers  in  SAHE-grantee  activities  are 
substantially  more  likely  to  report  enhanced  knowledge  in  technology  than  teachers  in  district 
activities  (50  percent  versus  24  percent),  and  somewhat  more  likely  to  report  enhanced  knowledge  in 
approaches  to  diversity  (35  percent  versus  26  percent). 

This  pattern  differs  somewhat  for  approaches  to  assessment:  48  percent  of  teachers  in  SAHE- 
grantee  activities  report  enhanced  knowledge  and  skills,  as  do  46  percent  for  teachers  in  district 
activities,  a negligible  difference.  We  are  not  sure  how  to  explain  the  lack  of  difference  between 
districts  and  SAHE  grantees  in  this  domain. 


EXHIBIT  3.15 

Percent  of  Teachers  Reporting  Enhanced  Knowledge  and  Skills  Due  to  Participation 
in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=731  to  750,  SAHE  Grantee  n=233  to  240) 
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In-depth  Knowledga  of  Curriculum  Instructional  Methods  Approaches  to  Use  oJ  Technology  Approaches  to 
Math/sctenca  Assessment  Diversity 

Areas  of  knowledge  and  skills 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower- assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  48  percent  of  the  teachers  who  participated  in  district  activities  report  their  in-depth  content 
knowledge  and  skills  have  been  enhanced  substantially  as  a result  of  professional  development.  Each  bar  and  the  number  on  the  top  of  it 
represent  the  percent  of  teachers  for  each  category. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the  program. 
“SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of  the  program. 
Due  to  missing  data,  the  district  n ranges  from  73 1 to  750;  the  SAHE  Grantee  n ranges  from  233  to  240, 


24  Analyses,  presented  in  detail  later  in  the  chapter,  show  that  the  difference  in  reported  outcomes  between  districts 
and  SAHE  grantees  can  be  explained  almost  entirely  by  the  fact  that  SAHE-grantee  activities  are  longer  and  give 
more  emphasis  to  content,  active  learning,  and  coherence. 
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We  averaged  each  teacher’s  responses  on  these  six  items  to  create  a composite  scale 
measuring  enhanced  knowledge  and  skills.  The  results  are  displayed  in  Exhibit  3.16.  One  fact 
immediately  visible  in  the  exhibit  is  the  substantial  variation  among  teachers  in  reported 
enhancement  in  knowledge  and  skills.  A few  teachers  have  composite  scores  of  one,  meaning  no 
change  in  knowledge  and  skills,  while  a few  have  composite  scores  of  five,  indicating  knowledge  and 
skills  were  enhanced  to  a great  extent  in  all  six  domains  measured.  The  average  composite  score  for 
teachers  participating  in  district  activities  is  3.2,  while  the  average  for  teachers  in  SAHE-grantee 
activities  is  3.6.  Our  results  also  indicate  that  the  average  enhancement  of  knowledge  and  skills  is 
somewhat  higher  for  teachers  participating  in  reform  activities  than  for  teachers  in  traditional 
activities. 


EXHIBIT  3.16 

Extent  to  Which  Participation  in  Eisenhower-assisted  Professional  Development 
Activities  Enhanced  Knowledge  and  Skills,  as  Reported  by  Teachers 
(District  n=750,  SAHE  Grantee  n=240) 


Overall  Traditional  Reform 

SAHE  Grantees 


Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that,  on  average,  those  teachers  who  participated  in  district  activities  reported  that  the 
extent  to  which  their  knowledge  and  skills  have  been  enhanced  in  six  different  areas  was  3.2,  where  zero  indicates  no  enhancement  and  five 
indicates  great  enhancement.  Compared  to  traditional  types  of  activities,  reform  types  were  significantly  higher  in  terms  of  enhancement  of 
knowledge  and  skills.  Each  dot  represents  one  teacher.  As  the  number  of  teachers  at  one  data  value  increases,  the  dots  form  a horizontal  line 
that  increases  in  length.  Each  distribution  represents  the  distribution  of  teachers  for  that  particular  category.  The  number  to  the  right  of  the 
distribution  is  the  mean. 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. 


Change  in  Classroom  Teaching  Practice 


We  conducted  a similar  analysis  of  changes  in  teaching  practice.  We  asked  the  teachers  in 
our  national  sample  to  what  extent  they  made  changes  in  their  teaching  practices  in  each  of  the 
following  domains,  as  a result  of  the  professional  development  activity:25 

♦ The  mathematics  curriculum  content 

♦ The  cognitive  challenge  of  math  classroom  activities 

♦ The  instructional  methods  employed 

♦ The  types  or  mix  of  assessments  used  to  evaluate  students 

♦ The  ways  technology  (calculator  or  computer)  is  used  in  instruction 

♦ The  approaches  taken  to  student  diversity. 

Teachers  were  asked  to  report  responses  on  a scale  from  0 to  3,  where  0=no  change,  l=minor 
change,  2=moderate  change,  and  3=significant  change. 

Exhibit  3.17  shows  that  the  results  are  generally  parallel  to  those  for  knowledge  and  skills. 
The  exhibit  displays  the  percent  of  teachers  responding  that  they  made  moderate  or  significant 
changes  in  their  teaching.  In  particular,  in  most  domains,  teachers  in  SAHE-grantee  activities  are 
more  likely  to  report  change  in  teaching  practice  than  teachers  in  district  activities.  However,  in  one 
domain — approaches  to  assessment — teachers  in  district  activities  are  more  likely  to  report  change  in 
practice  than  teachers  in  SAHE-grantee  activities. 


25  The  results  we  report  here  are  based  on  self-report  data.  In  our  third  report,  we  will  report  parallel  analyses  based 
on  the  longitudinal  study  of  teacher  change,  which  includes  data  on  teaching  practice  in  the  1996-97,  97-98,  and  98- 
99  school  years. 
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EXHIBIT  3.17 


Percent  of  Teachers  Reporting  Improvement  in  Classroom  Teaching  Practice  Due  to 
Participation  in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=731  to  750,  SAHE  Grantee  n=233  to  244) 
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Curriculum  Cognitive  Instructional  Approaches  to  Use  of  Approaches  to 

Challenge  Methods  Assessment  Technology  Diversity 

Types  of  teaching  practices 


■ Districts 

□ SAHE 
Grantees 


Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit.  The  first  bar  shows  that  46  percent  of  the  teachers  who  participated  in  district  activities  report  that  their  teaching 
practices  in  the  curriculum  content  area  have  improved  substantially  as  a result  of  participation  in  an  Eisenhower-assisted  professional 
development  activity.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each  category. 

Note.  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program.  Due  to  missing  data,  the  district  n ranges  from  73 1 to  750;  the  SAHE  Grantee  n ranges  from  233  to  240. 


We  averaged  each  teacher’s  responses  to  these  six  items  to  create  a composite  scale 
measuring  change  in  teaching  practice.  The  results  appear  in  Exhibit  3.18.  One  notable  pattern  is 
that  some  teachers  (about  17  percent  in  district  activities  and  10  percent  in  SAHE-grantee  activities) 
report  no  change  in  teaching  practice  in  any  of  the  domains  we  measured.  At  the  same  time,  some 
teachers  report  very  high  levels  of  change,  with  composite  scores  of  3 (the  maximum  value)  on  our 
scale. 
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EXHIBIT  3.18 


Degree  of  Improvement  in  Classroom  Teaching  Practice  Due  to  Participation  in 
Eisenhower-assisted  Professional  Development  Activities,  as  Reported  by  Teachers 

(District  n=767,  SAHE  Grantee  n=244) 
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Overall  Traditional  Reform 

SAHE  Grantees 


Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that,  on  average,  those  teachers  who  participated  in  district  activities  reported  that  the 
extent  of  their  improvement  in  classroom  teaching  practice  was  1.3,  where  zero  indicates  no  change  and  three  indicates  the  highest  degree  of 
change.  Each  dot  represents  one  teacher.  As  the  number  of  teachers  at  one  data  value  increases,  the  dots  form  a horizontal  line  that  increases 
in  length.  Each  distribution  represents  the  distribution  of  teachers  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is 
the  mean.  4 

Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower- assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component  of 
the  program. • 


The  average  composite  value  for  teachers  in  district  activities  is  1.3,  indicating  some  change 
in  teaching  practice,  while  the  average  value  for  teachers  in  SAHE-grantee  activities  is  1 .4. 

Although  this  difference  of  about  0.1  point  on  the  composite  scale  appears  small  in  numeric  terms,  it 
is  statistically  significant.  (The  standard  deviation  among  district  teachers  for  our  change  in  teaching 
composite  is  about  .8.  This  means  that  the  difference  between  teachers  in  SAHE-grantee  activities 
and  teachers  in  district  activities  is  about  .125  standard  deviations.)  The  difference  between 
traditional  and  reform  activities  is  somewhat  greater.  The  average  composite  score  is  1.4  for  teachers 
in  district  reform  activities  and  1 .2  for  district  traditional  activities,  a difference  of  0.2  point.  The 
parallel  results  for  teachers  in  SAHE-grantee  activities  are  1.6  and  1 .4,  also  a difference  of  0.2  point. 

Summary:  Teacher  Outcomes 

Overall,  our  results  indicate  that  a substantial  portion  of  teachers  in  SAHE-grantee  activities, 
and  a smaller  but  still  notable  proportion  of  teachers  in  district  activities,  report  that  participation 
enhanced  their  knowledge  and  skills.  For  example,  68  percent  of  teachers  participating  in  SAHE- 
grantee  activities  reported  enhanced  knowledge  in  mathematics  and  science,  compared  with  48 
percent  of  teachers  in  district  activities. 

A large  proportion  of  teachers  in  SAHE-grantee  activities  and  a somewhat  smaller  proportion 
in  district  activities  also  report  changes  in  teaching  practice,  as  a result  of  participation.  For  example. 
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66  percent  of  teachers  in  SAHE-grantee  activities  and  55  percent  of  teachers  in  district  activities 
report  making  changes  in  the  cognitive  challenge  of  classroom  activities  as  a result  of  participation. 

Our  data  indicate  that  teachers  participating  in  reform  types  of  activities,  such  as  mentoring, 
coaching,  and  study  groups,  are  somewhat  more  likely  than  teachers  in  traditional  forms  of 
professional  development,  such  as  workshops,  to  report  enhanced  knowledge  and  skills.  They  are 
also  more  likely  to  report  change  in  teaching  practice. 

With  respect  to  knowledge  and  skills,  the  effect  of  program  component  (SAHE  grantee 
versus  district)  is  larger  than  the  effects  of  activity  type  (reform  versus  traditional).  But,  with  respect 
to  change  in  teaching  practice,  the  effect  of  program  component  (SAHE  grantee  versus  district)  is 
smaller  than  the  effect  of  activity  type  (reform  versus  traditional).  This  may  be  a consequence  of  the 
fact  that  SAHE-grantee  activities  are  especially  likely  to  focus  on  subject  matter  content,  and  thus 
they  have  a strong  effect  on  teachers’  reported  change  in  knowledge  and  skills.  Reform  activities  are 
especially  likely  to  encourage  coherent  professional  development,  including  alignment  and 
professional  communication  among  teachers,  and  these  elements  may  be  important  supports  for 
change.  (This  issue  is  discussed  further  in  the  following  section.) 

Our  data  can  be  used  to  assess  Eisenhower  professional 
development  with  respect  to  the  Department  of  Education’s 
performance  indicator  1 . 1 (see  box),  pertaining  to  change  in 
teacher  knowledge  and  skills.  The  indicator  sets  a standard  of 
50  percent  of  teachers  reporting  enhanced  knowledge  and  skills 
as  a result  of  participation.  Our  results  indicate  that  district 
activities  meet  this  standard  for  two  domains  of  knowledge  and 
skills  (curriculum  and  instruction),  but  not  the  other  four  (in- 
depth  knowledge  in  mathematics  and  science,  approaches  to 
assessment,  uses  of  technology,  and  approaches  to  student 
diversity).  SAHE-grantee  activities  meet  the  standard  in  four  of 
the  six  domains  (in-depth  knowledge,  curriculum,  instruction, 
and  uses  of  technology). 

One  way  to  assess  the  magnitude  of  the  reported 
enhancement  of  knowledge  and  skills  is  to  compare  the  results 
we  obtained  with  the  results  for  professional  development 
activities  that  have  been  identified  as  exemplary.  Data  on 
enhancement  of  knowledge  and  skills  somewhat  parallel  to  ours 
are  available  for  teachers  participating  in  34  exemplary  summer 
institutes  in  mathematics  and  science,  supported  by  the  National  Science  Foundation,  the  Department 
of  Education,  and  other  agencies  (Carey  & Frechtling,  1997).  A comparison  of  our  data  for  SAHE- 
grantee  activities  with  the  results  obtained  for  the  34  exemplary  activities  indicates  that  teachers 
participating  in  SAHE-grantee  activities  report  enhancement  of  knowledge  and  skills  in  mathematics 
and  science  content  roughly  comparable  to  the  results  for  the  34  exemplary  activities.26  Teachers 


Indicator  1.1  Teachers' Skills 
and  Classroom  Instruction. 

By  1998,  over  50  percent  of  a 
sample  of  teachers  will  show 
evidence  that  participation  in 
Eisenhower  assisted 
professional  development  has 
resulted  in  an  improvement  in 
their  knowledge  and  skills,  and 
by  2000,  over  60  percent  will 
show  such  evidence.  By  1999, 
over  50  percent  of  a sample  of 
teachers  in  selected  sites  will 
show  evidence  that 
participation  in  Eisenhower- 
assisted  professional 
development  has  resulted  in 
improved  classroom 
instruction. 


25  Carey  and  Frechtling  (1997)  indicate  that  44  percent  of  participants  in  outstanding  teacher  development  programs 
reported  that  the  programs  enhanced  their  knowledge  and  understanding  of  science  content  to  “a  great  extent”  (value 
of  5 on  their  5-point  scale).  If  we  isolate  the  percentage  of  participants  in  SAHE-grantee  activities  who  reported  that 
the  activity  enhanced  their  mathematics  or  science  knowledge  “to  a great  extent”  (value  of  5 on  the  5-point  scale), 
the  percentage  is  41  percent.  The  comparable  percent  for  district  activities  is  24  percent. 
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participating  in  district  Eisenhower  activities  show  somewhat  weaker  results  than  do  teachers  in  the 
34  exemplary  activities. 


Participation  of  Teachers  from  High-Poverty  Schools 


Section  Findings 

♦ Teacher  participants  in  district  Eisenhower-assisted  activities  are  only  slightly  more 
likely  to  be  from  high-poverty  schools  than  are  teachers  in  the  national  teaching  force  as 
a whole.  Teacher  participants  in  SAHE-grantee  activities  are  less  likely  to  be  from  high- 
poverty  schools  than  are  teachers  in  the  national  teaching  force. 

The  Eisenhower  legislation  emphasizes  that  programs  and  activities  should  be  provided  to 
teachers  of  diverse  populations  of  students.  Several  provisions  of  the  law  state  that  funds  should  be 
used  to  provide  professional  development  that  benefits  students  from  diverse  backgrounds.  One  of 
the  law’s  purposes  is  to  incorporate 

...effective  strategies,  techniques,  methods,  and  practices  for  meeting  the  educational  needs 
of  diverse  student  populations,  including  females,  minorities,  individuals  with  disabilities, 
limited  English  proficient  individuals,  and  economically  disadvantaged  individuals,  in  order 
to  ensure  that  all  students  have  the  opportunity  to  achieve  challenging  State  student 
performance  standards  (Section  2002(1)(D)). 

In  addition,  the  local  plan  for  professional  development  is  required  to  describe  how  local 
professional  development  activities  will  meet  the  needs  of  teachers  of  diverse  student  populations. 

For  example,  applications  from  LEAs  are  required  to  include  a description  of  how  their  Title  II 
activities  “will  address  the  needs  of  teachers  in  schools  receiving  assistance  under  part  A of  Title  I” 
(Section  2208(d)(1)(B)). 

These  provisions  are  indicative  of  the  fact  that  teachers  in  high-poverty,  low-achieving 
schools  are  often  most  in  need  of  professional  development  (Darling-Hammond,  1997b).  Teachers  in 
high-poverty  schools  are  often  less  experienced  than  teachers  in  other  schools,  they  frequently  have 
students  who  are  more  challenging  to  teach,  and  they  often  face  larger  class  sizes  and  fewer  resources 
than  are  common  in  low-poverty  schools  (Darling-Hammond,  1997b;  U.S.  Department  of  Education, 
1999a).  In  addition,  teachers  in  high-poverty  schools  often  have  fewer  opportunities  to  participate  in 
certain  kinds  of  professional  development  than  their  counterparts  at  more  middle-class  schools  (U.S. 
Department  of  Education,  1998a). 
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To  estimate  the  percent  of  Eisenhower  participants  from  high-poverty  schools,  we  drew  on 
data  on  the  percent  of  students  eligible  for  the  free  lunch  in  each  of  the  schools  in  which  teachers  in 
our  national  sample  taught  at  the  time  of  the  survey.27  We  classified  schools  as  high-poverty  if  the 
percent  of  students  eligible  for  free  lunch  were  50  percent  or  greater.28  For  comparison,  we 
conducted  a parallel  analysis  of  the  percent  of  teachers  in  high-poverty  schools  for  the  full  national 
population  of  teachers. 

The  results  of  our  analysis  for  participants  in  district  Eisenhower  activities  are  displayed  in 
Exhibit  3.19.  The  initial  pair  of  bars  on  the  left  of  the  exhibit  indicate  that,  overall,  23  percent  of 
teachers  participating  in  district  Eisenhower  activities  are  from  high-poverty  schools,  as  compared  to 
about  21  percent  for  the  full  national  population  of  teachers.  This  difference  is  statistically 
significant,  indicating  that  participating  teachers  are  slightly  more  likely  to  be  from  high-poverty 
schools  than  are  teachers  in  the  nation  as  a whole  (pc.  10). 

In  part,  this  result  may  reflect  the  fact  that,  according  to  the  funding  formula  for  Title  II, 
funds  are  distributed  to  districts  based  in  part  on  the  districts’  Title  I allocations,  which  are  based  on 
the  number  of  school  children  in  poverty.  To  explore  the  extent  to  which  the  participation  of 
teachers  from  high-poverty  schools  is  due  to  the  funding  formula,  we  classified  the  districts  in  our 
sample  into  three  strata,  by  district  poverty.29 

Results  on  the  percent  of  Eisenhower  participants  from  high-poverty  schools  are  displayed 
separately  for  the  three  district  poverty  strata  in  Exhibit  3.19,  along  with  parallel  data  on  the  national 
percent  of  teachers  from  high-poverty  schools  in  each  of  these  strata.  The  data  indicate  that,  in  high- 
poverty  districts,  45  percent  of  the  nation’s  teachers  as  a whole  are  from  high-poverty  schools,  and 
51  percent  of  Eisenhower  participants  are  from  high-poverty  schools.  This  difference,  which  is 
statistically  significant,  suggests  that  high-poverty  districts  tend  to  concentrate  their  resources  on 
their  highest  poverty  schools.  In  medium-poverty  districts,  17  percent  of  Eisenhower  participants  are 
from  high-poverty  schools,  compared  with  14  percent  of  teachers  in  these  districts,  which  suggests 
somewhat  less  targeting  of  resources  within  medium-poverty  districts.  In  low-poverty  districts,  two 
percent  of  Eisenhower  participants  are  from  high-poverty  schools,  compared  with  four  percent  of 
teachers  in  these  districts. 


27  We  obtained  these  data  from  the  Common  Core  of  Data.  The  Common  Core  requests  information  on  the  number 
of  students  eligible  for  free  lunch  (but  not  reduced-price  lunch)  for  every  public  school  in  the  country. 

Unfortunately,  data  are  entirely  missing  from  some  states  and  for  some  schools  in  other  states.  We  used  a hot-deck 
imputation  method  to  impute  free-lunch  data  for  these  schools,  based  on  information  on  district  expenditures  for  free 
lunch,  as  well  as  information  on  school  level  (elementary,  middle,  and  high  school),  the  number  of  schools  in  the 
district,  and  the  minority  composition  of  the  school  in  relation  to  the  district  average.  Data  on  free-lunch 
participation  are  not  generally  available  for  private  schools,  and  thus  we  excluded  the  small  number  of  participating 
teachers  from  private  schools  from  this  analysis. 

28  To  eliminate  the  potential  impact  of  teacher  non-response  on  this  analysis,  we  included  all  teachers  in  our 
intended  sample,  whether  or  not  they  responded  to  the  survey. 

29  We  classified  districts  into  three  strata  according  to  the  percent  of  school-age  children  in  poverty,  based  on  the 
1990  Census.  We  used  these  strata  as  one  part  of  our  sampling  plan  for  districts.  See  Appendix  A. 
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EXHIBIT  3.19 

Percent  of  Teacher  Participations  in  District  Eisenhower-assisted  Activities 
Compared  to  All  Teachers  in  the  Nation,  Overall  and  by  District  Poverty  (n=1197) 


100% 

2 90%  - 

o 

o 

O 80%  - 


o 70%  - 

> 

o 


Overall  Low  Medium  High 


■ All  Teachers 
□Title  II  Participations 


District  poverty 


Source:  Results  for  all  teachers  are  based  on  data  on  the  full  population  of  schools  from  the  CCD.  Data  on  Title  II  Participations  are  based  on 
the  Mail  Survey  of  Teachers  Participating  in  Eisenhower- assisted  Telephone  Activities,  1998,  including  both  respondents  and  non -respondents. 
How  to  read  this  exhibit:  The  first  bar  shows  that  21  percent  of  teachers  are  in  high-poverty  schools.  The  second  bar  shows  that  23  percent  of 
participations  in  Eisenhower- assisted  activities  are  from  high-poverty  schools.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of 
teachers  for  each  category. 


We  carried  out  a parallel  analysis  of  the  percent  of  teachers  from  high-poverty  schools,  for 
teachers  participating  in  SAHE-grantee  activities.30  The  results,  shown  in  Exhibit  3.20,  indicate  that 
about  13  percent  of  SAHE-grantee  participants  are  from  high-poverty  schools,  which  is  significantly 
lower  than  the  overall  percent  of  teachers  in  high-poverty  schools  for  the  nation.  Thus,  S AHE 
grantees  appear  less  successful  in  targeting  their  activities  on  teachers  than  do  districts. 


30  Since  IHE/NPO  activities  are  not  necessarily  tied  to  particular  districts,  we  could  not  examine  the  results  by 
district  poverty. 
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EXHIBIT  3.20 


Percent  of  Teacher  Participations  in  District  and  SAHE  Grantee  Eisenhower-assisted 
Activities  from  High-poverty  Schools  (District  n=1 197,  SAHE  Grantee  n=254) 


All  teachers  District  participations  SAHE-grantee 

participations 


Group  of  teachers 


Source : Results  for  all  teachers  are  based  on  data  on  the  full  population  of  schools  from  the  CCD.  Data  on  Title  II  Participations  are  based  on 
the  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Telephone  Activities,  1998,  including  both  respondents  and  non-respondents. 
How  to  read  this  exhibit : The  first  bar  shows  that  21  percent  of  teachers  are  in  high-poverty  schools.  The  second  bar  shows  that  23  percent  of  in- 
district  Eisenhower-assisted  activities  are  from  high-poverty  schools.  The  third  bar  shows  that  13  percent  of  participations  in  SAHE-grantee 
activities  are  from  high-poverty  schools.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  for  each  category. 

Note : “District  participations”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE-grantee  participations”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 


Summary:  Participation  of  Teachers  from  High-poverty  Schools 

Our  results  indicate  that  teachers  in  district  Eisenhower- 
assisted  activities  are  slightly  more  likely  to  be  from  high-poverty 
schools  than  are  teachers  in  the  nation  as  a whole,  while  teachers  in 
SAHE-grantee  activities  are  somewhat  less  likely  to  be  from  high- 
poverty  schools.  Our  data  for  districts  indicate  that  the  standard  set 
in  the  Department  of  Education’s  Performance  Indicator  pertaining 
to  participation  from  high-poverty  schools  has  been  met  for  district 
activities.  (See  box.)  But  it  has  not  been  met  for  SAHE-grantee 
activities. 


To  some  extent,  the  results  for  districts  are  encouraging:  they  indicate  that  the  program  has 
been  somewhat  successful  at  targeting  resources  on  teachers  of  disadvantaged  children.  But  the 
percentage  of  teachers  from  high-poverty  schools  served  by  the  program  is  only  modestly  higher  than 
the  rate  for  the  nation,  indicating  that  the  more  progress  might  be  made  in  this  area.  We  give  more 


Indicator  4.1  High-poverty 
Schools.  The  proportion  of 
teachers  participating  in 
Eisenhower-assisted  activities 
who  teach  in  high-poverty 
schools  will  exceed  the 
proportion  of  the  national 
teacher  pool  who  teach  in  high- 
poverty  schools. 
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attention  to  this  issue  in  Chapter  4,  and  to  targeting  of  participants  in  SAHE-grantee  projects  in 
Chapter  6. 


What  Features  of  Professional  Development  Make  a 
Difference  for  Teacher  Outcomes? 


In  the  previous  sections  of  the  chapter,  we  have  described  the  quality  of  Eisenhower-assisted 
activities — as  defined  by  their  structural  and  core  features — and  teacher  outcomes  for  Eisenhower- 
assisted  activities,  as  reported  by  teachers  in  our  national  sample.  Now,  we  draw  on  these  data  to 
examine  the  relationships  among  these  characteristics  of  professional  development  activities,  in  an 
effort  to  identify  the  characteristics  of  professional  development  activities  that  are  related  to  teacher 
outcomes. 

Throughout  the  chapter,  we  have  characterized  professional  development  activities  in  terms 
of  structural  and  core  features.  We  view  the  three  structural  features — activity  type  (reform  versus 
traditional),  duration,  and  collective  participation — as  elements  that  set  the  parameters  or  context  in 
which  a professional  development  activity  takes  place.  And  we  view  the  three  core  features — 
content  focus,  active  learning,  and  coherence — as  characteristics  of  the  professional  development 
processes  and  experiences  that  take  place  during  an  activity.  Given  this  framework,  we  expect  the 
structural  features  of  professional  development  to  play  an  important  role  in  determining  the  substance 
or  core  of  the  professional  development  experienced  by  teachers;  and  we  expect  the  core  features  of 
the  professional  development  experienced  to  contribute  to  teacher  outcomes,  including  enhanced 
knowledge  and  skills  and  improvements  in  teaching  practice. 

To  test  this  view  of  professional  development,  we  estimated  a formal  causal  model,  using 
data  from  our  national  sample  of  teachers.  The  model  serves  two  related  purposes:  it  enables  us  to 
determine  whether  the  relationships  we  have  hypothesized  are  supported  by  the  data  we  have 
collected,  and  it  also  permits  us  to  develop  estimates  of  the  strength  of  these  relationships.  The 
model  incorporates  the  following  measures  of  the  structural  features  of  professional  development: 
activity  type  (coded  reform=l,  traditional=0);  two  aspects  of  duration — time  span  (coded  l=less  than 
one  day,  6=more  than  one  month)  and  number  of  contact  hours;  and  collective  participation  (coded 
0=not  collective,  l=somewhat,  2=collective).:u 

The  model  also  incorporates  the  following  measures  of  the  core  features  of  the  professional 
development  experiences:  content  focus  (coded  0=not  at  all,  l=minor  emphasis,  2=major  emphasis); 
active  learning  (coded  as  the  number  of  types  of  opportunities  for  active  learning,  from  0 to  20);  and 
coherence  (coded  as  the  number  of  elements  of  coherence,  from  0 to  9). 

Finally,  the  model  includes  the  following  outcome  measures:  enhanced  knowledge  and  skills 
(coded  from  l=not  at  all  to  5=great  extent);  and  change  in  teaching  practice  (coded  from  0=no 
change  to  3=significant  change). 

Since  it  is  possible  that  teachers  in  different  types  of  schools  or  teachers  with  different 
characteristics  may  experience  different  types  of  professional  development,  we  have  included  school 


31  See  Appendix  E for  more  a more  detailed  description  of  the' variables. 
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and  teacher  characteristics  as  control  variables  in  our  model.  The  model  includes  the  characteristics 
of  the  schools  in  which  the  participating  teachers  teach:  the  percent  of  students  eligible  for  free  lunch 
and  the  percent  of  minority  enrollment.  The  model  also  includes  five  characteristics  of  the 
participating  teachers:  gender,  subject  of  the  teacher’s  professional  development  experience 
(mathematics  or  science);  grade  level  (elementary,  middle,  or  high  school);  whether  the  teacher  is 
certified  in  the  teacher’s  main  teaching  field;  and  the  teacher’s  teaching  experience,  in  years.32  (See 
Appendix  E for  more  information  on  our  measures  of  these  school  and  teacher  characteristics.) 

In  addition,  since  we  are  interested  in  understanding  the  differences  between  activities 
supported  through  the  district  and  SAHE  component  of  the  program,  we  have  also  included  the 
sponsorship  of  the  activity  as  a variable  in  the  model  (coded  1=SAHE  grantee,  0=district). 

The  results  are  shown  in  Exhibit  3.21.  (A  more  detailed  presentation  of  the  results  appears  in 
Appendix  E.)  The  results  shown  are  expressed  as  standardized  path  coefficients,  which  represent  the 
relative  influence  of  one  variable  on  another.  Path  coefficients  generally  have  values  between  -1  and 
+1,  and  coefficients  with  larger  absolute  values  indicate  stronger  relationships.  All  paths  shown  are 
statistically  significant. 

To  interpret  the  results,  we  begin  by  discussing  the  variables  at  the  left-hand  side 
(sponsorship  and  structural  features)  and  proceed  to  discuss  the  core  features  and  teacher  outcomes. 
First,  the  analysis  indicates  that  sponsorship  has  a substantial  effect  on  both  the  time  span  and  contact 
hours  spent  in  professional  development:  on  average,  SAHE-grantee  activities  are  spread  over  a 
substantially  longer  period  of  time  and  involve  more  hours  of  professional  development  than  district 
activities.33  Sponsorship  also  has  an  influence  on  active  learning  and  collective  participation: 
SAHE-grantee  activities  are  more  likely  to  provide  opportunities  for  active  learning,  and  less  likely 
to  be  designed  for  collective  participation  than  district  activities;  that  is,  they  are  less  likely  to  be 
designed  for  teachers  in  school,  department,  or  grade-level  groupings.  Finally,  sponsorship  has  a 
direct  influence  on  content  focus:  controlling  for  time  span  and  contact  hours,  SAHE-grantee 
activities  tend  to  give  more  emphasis  to  content  than  do  district  activities. 

Activity  type  is  somewhat  less  influential  than  sponsorship,  but,  like  sponsorship,  it  has  an 
important  influence  on  duration:  reform  activities  tend  to  span  longer  periods  and  to  involve  greater 
numbers  of  contact  hours  than  traditional  activities. 


32  With  few  exceptions,  we  found  few  systematic  differences  in  Eisenhower  professional  development  experiences 
for  teachers  in  different  types  of  schools  or  with  different  characteristics.  One  teacher  characteristic  that  has  a 
consistent  effect  is  grade  level  taught.  Teachers  in  secondary  schools  tend  to  report  participating  in  activities  with 
less  positive  quality  (for  example,  fewer  opportunities  for  active  learning  and  less  change  in  teaching  practice).  See 
Appendix  E. 

33  These  and  all  other  results  described  control  for  the  school  and  teacher  characteristics  discussed  in  the  text. 
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As  we  expected,  our  two  measures  of  duration — time  span  and  amount  of  time — exert  a 
substantial  influence  on  the  core  features  of  professional  development  experiences.34  Time  span  and 
amount  of  time  have  a substantial  positive  influence  on  opportunities  for  active  learning  and 
coherence.  Longer  activities  tend  to  include  substantially  more  opportunities  for  active  learning 
(such  as  the  opportunity  to  plan  for  classroom  implementation,  observe  and  be  observed  teaching, 
review  students’  work,  and  give  presentations  and  demonstrations);  and  they  also  tend  to  incorporate 
more  aspects  of  coherence  (including  connections  to  a teacher’s  goals  and  experiences,  alignment 
with  standards,  and  professional  communication  with  other  teachers).  Time  span  and  amount  of  time 
also  have  a moderately  positive  influence  on  the  emphasis  given  to  content  knowledge:  activities 
that  span  a longer  period  and  last  more  hours  are  more  likely  to  focus  on  mathematics  and  science 
content. 


The  fact  that  both  time  span  and  contact  hours  have  independent  effects  on  our  measures  of 
core  features  suggests  that  both  dimensions  of  duration  are  important.  Professional  development  is 
likely  to  be  of  higher  quality  if  it  is  both  sustained  over  time  and  involves  a substantial  number  of 
hours. 


As  we  anticipated,  all  three  of  our  measures  of  the  core  features  of  activities  have  a positive 
influence  on  enhanced  knowledge  and  skill,  as  reported  by  the  teachers  in  our  sample.  Both  content 
focus  and  coherence  have  substantial  positive  effects  on  enhanced  knowledge  and  skills,  indicating 
that  activities  that  give  greater  emphasis  to  content  and  that  are  better  connected  to  teachers’  other 
professional  development  experiences  and  other  reform  efforts  are  more  likely  to  produce  enhanced 
knowledge  and  skills.35  Active  learning  is  also  related  to  enhanced  knowledge  and  skills,  but  the 
effect  is  less  strong. 

Finally,  enhanced  knowledge  and  skills  have  a substantial  positive  influence  on  change  in 
teaching  practice:  teachers  who  report  enhanced  knowledge  and  skills  are  likely  to  report  changing 
their  teaching  practices  as  well.  In  addition,  the  coherence  of  professional  development  activities  has 
an  important  positive  influence  on  change  in  teaching  practice,  over  and  above  the  effects  of 
knowledge  and  skills.  This  suggests  that  teachers  who  experience  professional  development  that  is 
connected  to  their  other  professional  development  experiences,  is  aligned  with  standards  and 
assessments,  and  fosters  professional  communication,  are  more  likely  to  change  their  practice,  even 
among  teachers  who  have  gained  the  same  underlying  knowledge  and  skills  as  a result  of  their 
professional  development  experiences. 

These  results  are  very  encouraging,  for  a number  of  reasons.  First,  they  tend  to  confirm 
several  of  the  key  assumptions  underlying  the  Title  II  reauthorizing  legislation.  For  example,  our 
results  indicate  that  sustained  and  intensive  professional  development  is  more  likely  to  be  of  high 
quality,  as  reported  by  teachers,  than  is  shorter  professional  development.  Our  results  also  indicate 


34  The  model  predicting  the  core  features  and  teacher  outcomes  includes  sponsorship,  the  structural  features,  and  all 
control  variables. 

35  When  enhancements  in  knowledge  and  skills  is  controlled,  content  focus  has  a negative  association  with  changes 
in  classroom  practice.  We  suspect  that  this  result  is  probably  spurious,  resulting  from  the  large  number  of 
independent  variables  included  in  the  model  predicting  change  in  teaching  practice.  (The  model  for  change  in 
teaching  practice  includes  all  of  the  structural  and  core  features  of  professional  development,  as  well  as  all  control 
variables.)  We  estimated  a model  predicting  change  in  teaching  practice  but  omitting  knowledge  and  skills  as  an 
intervening  variable,  and,  in  that  model,  content  focus  has  a positive  effect. 
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that  professional  development  that  focuses  on  academic  subject  matter,  gives  teachers  opportunities 
for  “hands-on”  work,  and  is  integrated  into  the  daily  life  of  the  school,  is  more  likely  to  produce 
enhanced  knowledge  and  skills. 

Second,  the  results  provide  insight  into  the  reasons  why  teachers  in  SAHE-grantee  activities 
tend  to  report  more  positive  outcomes  than  teachers  in  district  activities.  In  particular,  the  model 
indicates  that  the  difference  between  SAHE-grantee  and  district  activities  can  be  accounted  for 
almost  entirely  by  the  fact  that  SAHE-grantee  activities  are  of  substantially  longer  duration  than 
district  activities  (both  in  terms  of  time  span  and  contact  hours),  and  they  give  more  emphasis  to 
subject  matter  content.  Our  results  indicate  that,  when  these  structural  and  core  features  are 
controlled,  sponsorship  (SAHE  grantee  versus  district)  does  not  have  a direct  effect  on  teacher 
outcomes. 

In  addition,  our  results  indicate  that  the  effects  of  activity  type  are  also  largely  indirect, 
operating  through  duration.  Reform  activities  tend  to  be  of  longer  duration  than  traditional  activities, 
and  this  accounts  for  nearly  all  of  the  positive  effects  of  type.36 

Finally,  our  results  call  attention  to  the  importance  of  the  coherence  of  professional 
development  activities.  Activities  that  are  linked  to  teachers’  other  experiences,  aligned  with  other 
reform  efforts,  and  encouraging  of  professional  communication  among  teachers  appear  to  support 
change  in  teaching  practice,  even  after  the  effects  of  enhanced  knowledge  and  skills  are  included  in 
the  model. 


Summary  and  Conclusions 


In  this  chapter,  we  have  drawn  on  our  nation  sample  of  teachers  who  participated  in 
Eisenhower-assisted  activities  to  undertake  a number  of  analyses.  First,  we  have  drawn  on  the  data 
to  describe  the  characteristics  of  these  activities  and  to  assess  the  extent  to  which  these  characteristics 
reflect  the  intent  of  the  authorizing  legislation  and  the  descriptions  of  best  practice  in  the  literature. 
Second,  we  have  drawn  on  the  data  to  examine  the  degree  to  which  Eisenhower-assisted  activities 
have  resulted  in  enhanced  knowledge  and  skills  and  improved  teaching  practices,  as  reported  by  the 
teachers  in  our  sample.  We  have  used  our  data  to  identify  the  relationship  between  the  characteristics 
of  Eisenhower-assisted  activities  and  outcomes  for  teachers. 

Our  results  indicate  that,  on  the  average,  supported  activities  share  some  but  not  all  of  the 
characteristics  of  high-quality  professional  development  identified  in  the  literature.  Activities 
supported  through  the  SAHE  component  of  the  program  tend,  on  average,  to  be  of  substantial 
duration,  and  many  SAHE-grantee  activities  focus  on  mathematics  and  science  content  and  provide 
appropriate  active  learning  opportunities  for  teachers.  Some  activities  supported  by  the  district 
component  of  the  program  also  share  these  elements  of  high  quality,  but,  on  average,  district 
activities  tend  to  be  somewhat  less  consistent  with  the  dimensions  of  high  quality  identified  in  the 
literature. 


36  Our  results  show  a modest  direct  effect  of  activity  type  on  enhanced  knowledge  and  skills,  indicating  that  reform 
activities  have  slightly  more  positive  outcomes  when  all  of  the  design  features  and  quality  characteristics  in  our 
model  are  included.  , 
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Many  teachers  who  participate  in  Eisenhower-assisted  activities  report  enhanced  knowledge 
and  skill  as  a result  of  participation,  as  well  as  changes  in  teaching  practice.  This  is  especially  true 
for  SAHE-grantee  activities;  somewhat  fewer  teachers  in  the  district  component  of  the  program 
report  gains  in  knowledge  and  skills  or  changes  in  teaching  practice. 

Our  analysis  of  the  relationship  between  the  characteristics  of  Eisenhower-assisted  activities 
and  teacher  outcomes  indicates  that  several  structural  and  core  features  play  a strong  role  in 
influencing  teacher  outcomes.  In  particular,  the  duration  of  activities  has  a strong  influence  on  the 
degree  to  which  the  activities  are  able  to  provide  high-quality  experiences  for  teachers.  Both  the  time 
span  over  which  activities  are  spread  and  the  number  of  contact  hours  of  professional  development 
provided  have  a substantial  independent  influence  on  the  core  features  of  the  professional 
development  provided.  In  addition,  the  organizational  form  or  type  of  the  activity  also  has  an 
influence  on  the  core  features  of  the  experience  provided.  On  average,  reform  types  of  activities, 
such  as  mentoring  and  study  groups,  tend  to  provide  higher  quality  experiences  than  traditional 
activities,  such  as  workshops,  although  there  is  a substantial  range  of  experiences  within  each  type. 

Our  data  indicate  that  three  core  features  of  professional  development  have  an  important 
influence  on  the  extent  to  which  participation  enhances  teachers’  knowledge  and  skills  and  leads  to 
improved  teaching  practices,  as  reported  by  teachers  in  our  sample.  In  particular,  positive  teacher 
outcomes  are  related  to  the  extent  to  which  activities  focus  on  subject  matter  content,  provide  active 
learning  opportunities  for  teachers,  and  nurture  a coherent  program  of  professional  development  that 
is  connected  to  other  aspects  of  teachers’  work  lives. 

Finally,  the  features  of  professional  development  we  have  identified  in  our  model  help 
explain  why  SAHE-grantee  activities  have  more  positive  effects,  as  reported  by  teachers,  than  district 
activities.  SAHE-grantee  activities  tend  to  have  longer  time  spans  and  provide  more  hours  of 
professional  development  than  district  activities,  and  this  in  turn  permits  SAHE-grantee  activities  to 
give  more  emphasis  to  subject  matter  content,  to  provide  more  active  learning  opportunities,  and  to 
strengthen  the  coherence  of  teachers’  professional  development  experiences. 

To  build  on  this  analysis  of  teachers’  report  of  their  experiences  in  Eisenhower-supported 
activities,  the  next  chapter  turns  to  an  analysis  of  district  coordinators’  descriptions  of  their 
Eisenhower-supported  activities. 
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CHAPTER  4 


DISTRICT  “PORTFOLIOS”  OF  EISENHOWER-ASSISTED 
PROFESSIONAL  DEVELOPMENT  ACTIVITIES 


School  districts  shape  the  use  of  the  majority  of  Eisenhower  Professional  Development 
Program  (EPDP)  funds;  eighty-four  percent  of  Eisenhower  funds  go  to  the  district  component  of  the 
program.1  In  the  last  chapter,  we  described  patterns  of  teachers’  participation  in  Eisenhower-assisted 
professional  development  activities,  including  teachers’  reports  of  the  characteristics  and  qualities  of 
those  activities,  and  how  the  activities  increased  teachers’  knowledge  and  skills  and  changed  their 
teaching  practice.  Now  we  turn  to  the  district’s  role  in  shaping  teachers’  professional  development 
experiences.  Teachers’  experiences  in  Eisenhower-assisted  professional  development  activities 
depend  largely  on  two  things:  (1)  the  types  of  professional  development  activities  districts  make 
available  to  teachers,  and  (2)  how  teachers  come  to  participate  in  these  activities.  In  this  chapter  we 
look  at  the  mix  of  professional  development  activities  that  districts  support  with  Eisenhower 
funds — their  “portfolio”  of  Eisenhower-assisted  activities — and  the  selection  of  teachers  to 
participate  in  these  activities. 

Provisions  of  the  authorizing  legislation  guide  district  decisions  about  the  characteristics  of 
Eisenhower-assisted  professional  development  activities  and  who  participates  in  them.  The 
Elementary  and  Secondary  Education  Act,  as  amended  by  the  Improving  America’s  Schools  Act  of 
1994,  describes  the  characteristics  that  districts  should  strive  to  incorporate  in  the  professional 
development  opportunities  that  they  provide.  In  particular,  the  legislation  aims  to  support  “intensive, 
ongoing  professional  development  programs”  (Section  2207(5)(A))  that  include  “sustained  and 
intensive  high-quality  professional  development”  (Section  2101(a)(1)).  Congress  further  directs  that 
local  Eisenhower  plans  be  designed  in  ways  that  would  likely  affect  teacher  practice  and  “have  a 
positive  and  lasting  impact  on  the  student’s  performance  in  the  classroom”  (Section  2208(d)(1)(E)). 

The  legislation  also  provides  that  Eisenhower-assisted  activities  should  address  the  needs  of 
teachers  of  students  from  historically  underrepresented  groups  (Section  2205(b)(2)(F)).  In  particular, 
because  of  Title  I’s  size  and  prominence  in  serving  children  at  risk  of  school  failure,  the  Eisenhower 
legislation  places  a special  emphasis  on  addressing  the  needs  of  teachers  in  schools  receiving  Title  I, 
Part  A funds  (Sections  2205(b)(2)(E),  2208(b)(2),  and  2208(d)(1)(B)).  The  Title  I statute  has  a 
similar  provision  regarding  the  Eisenhower  Professional  Development  Program  (Section 
1 1 19(b)(l  1)(C)). 

In  this  chapter,  we  describe  how  school  districts  differ  from  one  another  in  the  types  of 
activities  that  they  support  with  Eisenhower  funds.  We  examine  how  these  activities  differ  on 
several  of  the  dimensions  of  the  structural  and  core  features  that  were  related  to  teacher  outcomes  in 
Chapter  3.  We  also  discuss  district  practices  of  targeting  specific  groups  of  teachers,  and  the  ways 
that  districts  select  teachers  to  participate  in  the  activities.  Exhibit  4.0  highlights  how  the  issues  that 
we  address  in  this  chapter  fit  into  the  framework  of  the  entire  report. 

1 State  education  agencies  (SEAs)  receive  84  percent  of  the  total  Title  II  allocation  (16  percent  goes  to  State 
Agencies  of  Higher  Education).  Of  that  84  percent,  SEAs  are  required  to  pass  on  at  least  90  percent  of  the  funds  to 
districts;  states  have  the  option  of  using  five  percent  of  the  funds  for  administration  and  five  percent  for  their  own 
programs. 
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EXHIBIT  4.0 


Conceptual  Framework  for  This  Evaluation 


While  the  last  chapter  described  teachers’  experiences  in  Eisenhower-assisted  professional 
development  activities,  this  chapter  turns  to  similarities  and  differences  across  districts  in  the 
professional  development  activities  that  districts  support.  And  while  the  previous  chapter  described 
the  characteristics  of  teachers  nationwide  who  participate  in  Eisenhower-assisted  activities,  this 
chapter  describes  the  similarities  and  differences  across  districts  in  targeting  and  recruiting  teachers 
to  participate  in  Eisenhower-assisted  activities. 

Data  Sources 

We  rely  heavily  in  this  chapter  on  National  Profile  data  from  our  telephone  surveys  of  a 
national  probability  sample  of  district  Eisenhower  coordinators.  Where  appropriate,  we  supplement 
these  survey  data  with  case-study  information  regarding  district  patterns  of  Eisenhower  support  for  a 
variety  of  different  types  of  professional  development  opportunities  and  experiences.  Our  case-study 
data  come  from  two  sources.  One  source  is  a series  of  in-depth  case  studies  that  we  conducted 
during  the  1997-1998  school  year.  We  chose  10  districts,  two  from  each  of  five  states,  to  allow 
variation  on  state-level  reform  efforts,  the  district’s  approach  to  professional  development,  and 
demographic  and  geographic  characteristics.  The  second  source  of  case-study  data  is  a series  of  six 
exploratory  case  studies  that  we  conducted  during  the  spring  of  1997,  also  chosen  to  capture 
variation  on  these  dimensions. 

To  obtain  the  National  Profile  data  we  conducted  telephone  interviews  with  a national 
probability  sample  of  district  Eisenhower  coordinators  in  the  spring  of  1998.  Through  a system  of 
stratified  sampling  to  ensure  variation  on  district  poverty  level,  we  randomly  drew  about  400 
districts,  giving  larger  districts  a higher  chance  of  being  drawn;  of  these  400,  we  were  able  to 


"*  • 4-2 


BEST  COPY  AVAILABLE 


1 63 


interview  363,  which  provides  us  with  a response  rate  of  88  percent  (see  Appendices  A and  B for 
more  details  about  our  National  Profile  and  case-study  design). 

During  the  telephone  interviews,  district  coordinators  reported  on  professional  development 
activities  that  occurred  during  the  time  period  from  July  1 through  December  31,  1997.  As  a result, 
the  data  referenced  and  exhibited  in  this  chapter  that  represent  characteristics  of  specific  professional 
development  activities  refer  to  activities  that  took  place  during  this  time  period.  Questions  that  do  not 
refer  to  specific  activities,  but  to  general  practices  (e.g.,  targeting  groups  of  teachers),  apply  to  the 
entire  1997-1998  school  year.  Ten  districts  in  our  sample  report  that  they  offered  no  activities  over 
the  period  from  July  1 through  December  31,1 997 ; therefore,  analyses  in  this  chapter  that  apply  to 
particular  activities  exclude  these  ten  districts,  and  thus  are  conducted  with  a maximum  sample  of 
353  districts. 

The  probability  of  a district  being  selected  into  our  national  sample  was  proportional  to  the 
number  of  teachers  in  the  district.  Consequently,  all  of  the  results  are  weighted  by  the  size  of  the 
district  (i.e.,  the  number  of  teachers  in  the  district).  As  a result,  our  data  reflect  information 
according  to  the  percent  of  teachers  in  a district.  For  questions  that  ask  about  teacher  participation  in 
Eisenhower-assisted  activities,  we  report  the  number  of  participations  rather  than  the  number  of 
participants.  As  a rule,  districts  are  unable  to  determine  whether  Eisenhower  participants  attended 
multiple  Eisenhower-assisted  activities.  Therefore,  a single  participant  may  account  for  more  than 
one  “participation.” 

Organization  of  Chapter 

In  this  chapter,  we  examine  district  “portfolios”  of  Eisenhower-assisted  professional 
development  activities.  The  mix  of  professional  development  activities  that  a district  supports  with 
Eisenhower  funds  can  be  viewed  in  its  entirety  as  a “portfolio”  of  Eisenhower-assisted  professional 
development  activities.  Activity  portfolios  can  differ  according  to  the  types  and  range  of 
opportunities  offered,  as  well  as  according  to  the  structure  and  substance,  or  core,  of  the 
opportunities. 

For  example,  district  portfolios  of  professional  development  activities  can  place  more  or  less 
emphasis  on  particular  subject  areas;  they  can  include  several  different  kinds  of  activities  or  be 
limited  to  only  one  or  two;  and  they  can  place  more  or  less  emphasis  on  strategies  that  afford 
teachers  the  time  to  learn  complex  subject  matter  and  to  reflect  on  and  practice  what  they  have 
learned.  Taken  together,  the  activities  that  comprise  the  portfolios  of  professional  development 
activities  represent  a district’s  professional  development  strategy,  although  the  degree  to  which 
districts  strategically  plan  their  portfolios  varies  from  district  to  district. 

To  describe  the  district  Eisenhower  portfolio,  we  divide  the  chapter  into  five  main  sections. 
The  first  section  examines  the  subject  area  focus  of  Eisenhower-assisted  professional  development 
activities.  The  second  section  describes  districts’  patterns  of  support  for  “traditional”  versus 
“reform”  types  of  professional  development  activities,  and  the  other  structural  and  core  features  of 
these  activities.  Workshops  and  conferences  are  considered  to  be  “traditional”  forms  of  professional 
development.  Activities  that  appear  to  be  structured  to  allow  longer  duration  and  greater  depth  and 
focus,  such  as  mentoring  or  committee  or  task  force  participation,  are  considered  to  be  “reform” 
activities  (Little,  1993;  Sparks  & Loucks-Horsley,  1989).  (See  Chapter  3 for  a detailed  discussion  of 
types  of  activities.)  The  structural  and  core  features  discussed  in  these  sections  include  the  duration 
of  the  activity,  both  in  number  of  contact  hours  and  span  of  time  across  days,  weeks,  or  months; 
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collective  participation,  or  the  extent  to  which  groups  of  teachers  or  whole  schools  participate 
together  in  the  activity;  and  the  types  of  active  learning  opportunities  that  the  activities  provide  to 
teachers.  As  part  of  our  examination  of  district  portfolios  in  this  section,  we  also  compare 
Eisenhower-assisted  activities  in  the  district  to  the  district’s  entire  program  of  professional 
development. 

In  the  third  section  of  the  chapter,  we  examine  district  strategies  concerning  targeting  and 
recruiting  teachers  into  Eisenhower-assisted  professional  development  activities.  There  is  sometimes 
greater  need  for  professional  development  for  teachers  in  high-poverty  areas,  and  often  district 
strategies  for  recruiting  these  teachers  for  participation  in  professional  development  meet  with 
limited  success.  In  Chapter  3,  we  showed  that  teachers  from  high-poverty  schools  are  only 
somewhat  more  likely  to  participate  in  Eisenhower-assisted  professional  development  than  other 
teachers.  Here  we  examine  district  strategies  that  may  explain  these  participation  rates.  To  address 
these  targeting  and  participation  issues,  we  focus  on:  1)  the  targeting  of  Eisenhower-assisted 
activities  toward  special  populations  of  teachers,  2)  how  teachers  come  to  participate  in  Eisenhower- 
assisted  activities  (e.g.,  whether  they  volunteer  or  are  selected),  and  3)  strategies  that  districts  use  to 
increase  participation. 

Throughout  the  chapter,  we  report  where  the  structural  and  core  features  of  district 
professional  development,  and  district  targeting  patterns  vary  significantly  by  district  poverty  level 
(defined  as  the  number  of  children  living  in  poverty  in  the  district)  and  the  size  of  the  district 
(defined  as  the  number  of  teachers  in  a district),  or  both.  Differences  among  variables  by  size  and/or 
poverty  are  reported  if  they  are  statistically  significant  at  the  .05  level.  For  these  analyses,  both 
district  poverty  and  district  size  are  always  estimated  in  the  same  model,  so  effects  for  one  always 
control  for  the  effects  of  the  other.  Therefore,  any  significant  size  effects  are  independent  of  poverty 
effects,  and  likewise  any  significant  poverty  effects  are  independent  of  size  effects.  Whenever  we 
test  for  poverty  and  size  differences,  we  report  the  findings.  If  there  are  significant  effects  either  by 
district  poverty  level,  by  district  size,  or  both,  we  show  this  in  an  exhibit;  if  the  effects  of  both  are 
insignificant,  we  report  the  findings  in  the  text,  but  do  not  include  an  exhibit.  Interaction  effects 
between  poverty  and  size  are  insignificant  unless  otherwise  noted.2 

We  divide  poverty  into  three  levels — low  (less  than  10.9  percent  of  children  in  poverty), 
medium  (from  10.9  to  21.4  percent  of  children  in  poverty),  and  high  (greater  than  21.4  percent  of 
children  in  poverty).3  District  size  is  divided  into  four  types — small  (districts  with  less  than  250 
teachers),  medium  (districts  with  between  250  and  1500  teachers),  large  (districts  with  greater  than 
1500  teachers),  and  consortia.  A consortium  is  a group  of  districts,  ranging  in  size  from  several 
districts  to  several  hundred  districts,  which  can  sometimes  comprise  a substantial  portion  of  a state. 
To  identify  consortium  status,  we  asked  each  sampled  district  whether  or  not  the  district  participated 
in  the  Eisenhower  program  through  a consortium.  If  the  district  indicated  that  the  district 
participated  through  a consortium,  we  then  drew  the  entire  consortium  into  our  sample,  and  adjusted 
the  probability  of  the  consortium  being  selected  into  the  sample,  based  on  the  full  set  of  member 
districts.  In  reporting  results,  we  use  “district”  to  indicate  district  or  consortium,  unless  otherwise 
noted. 

2 Means  and  standard  deviations  for  all  of  the  variables  analyzed  in  this  chapter  are  located  in  Appendix  F,  listed  by 
exhibit  number.  All  parameter  estimates  reported  in  the  chapter  incorporate  weights  reflecting  the  sampling  plan. 
Reported  p-values  and  the  standard  errors  on  which  they  are  based,  however,  do  not  reflect  the  stratification  and 
variance  in  weights  incorporated  in  the  design.  Analyses  that  take  these  elements  of  the  complex  sample  design  into 
account  have  been  carried  out,  and  the  results  are  nearly  identical  to  those  reported  in  the  chapter. 

3 These  categories  divide  the  population  equally  into  thirds. 
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In  the  fourth  section  of  the  chapter,  we  summarize  and  synthesize  the  findings  for  district 
poverty  and  district  size.  We  discuss  how  the  level  of  poverty  and  the  number  of  teachers  in  a district 
might  affect  district  Eisenhower  portfolios,  and  targeting  and  recruiting  strategies,  and  discuss  the 
implications  of  these  findings  for  the  Eisenhower  program.  The  fifth  and  final  section  of  the  chapter 
summarizes  our  major  findings  about  district  portfolios  of  Eisenhower-assisted  professional 
development,  and  discusses  implications  for  district  and  federal  policy. 


District  Portfolios’  Emphasis  on  Mathematics  and  Science 


Section  Findings 

♦ Nearly  all  teachers  are  in  districts  that  use  Eisenhower  funds  for  professional 

development  activities  in  mathematics  and  science.  Although  a substantial  number  of 
districts  use  Eisenhower  funds  for  activities  outside  of  mathematics  and  science, 
particularly  language  arts  and  social  studies,  teacher  participation  is  still  mainly  focused 
on  mathematics  and  science  activities. 

Like  its  predecessor,  the  Eisenhower  Mathematics  and  Science  Education  Program,  the 
reauthorized  Eisenhower  Professional  Development  Program  focuses  primarily  on  supporting 
professional  development  in  mathematics  and  science.  But  the  1994  reauthorization  changed  the  law 
to  allow  districts  to  use  some  portion  of  their  Eisenhower  funds  to  support  professional  development 
in  subject  areas  outside  of  mathematics  and  science.  Although  the  main  focus  of  program-funded 
activities  was  to  remain  mathematics  and  science,  appropriated  funds  in  excess  of  $250  million  could 
be  targeted  to  other  subject  areas  (Section  2206).  During  the  1997-98  school  year,  22.7  percent  of  a 
district’s  Eisenhower  funds  could  go  to  support  professional  development  in  other  subject  areas 
besides  mathematics  and  science.  Furthermore,  states  and  districts  could  apply  for  waivers  from  the 
federal  government  to  allow  them  to  devote  larger  percentages  of  their  Eisenhower  grants  to 
professional  development  in  other  subject  areas.  Ten  states  and  two  districts  have  been  granted  such 
waivers.  In  addition,  states  may  apply  to  the  Department  of  Education  for  “ED  Flex”  status,  which  if 
granted,  allows  states  to  grant  waivers  to  LEAs  that  request  them.  These  waivers  may  apply  to  Title  II  or 
to  other  federal  programs.  As  of  January  1998,  twelve  states  had  been  granted  Ed  Flex  status  by  ED: 
Colorado,  Illinois,  Iowa,  Kansas,  Maryland,  Massachusetts,  Michigan,  New  Mexico,  Ohio,  Oregon, 

Texas,  and  Vermont.  Two  of  our  case  districts  are  located  in  ED  Flex  states — Ohio  and  Texas. 

As  with  other  parts  of  the  ESEA,  the  intent  of  these  provisions  was  to  allow  school  districts 
flexibility  in  using  Eisenhower  funds;  to  allow  professional  development  to  be  responsive  to  the 
National  Education  Goals  that  call  for  students  to  demonstrate  competence  in  all  major  subject  areas; 
and  for  professional  development  to  prepare  teachers  to  instruct  students  in  these  subject  areas. 
Expanding  the  Eisenhower  program  to  other  subject  areas  besides  mathematics  and  science  was  also 
designed,  according  to  ED  officials,  to  allow  districts  to  keep  pace  with  the  standards-based  reform 
movement,  which  has  fostered  the  development  of  standards  in  all  major  subject  areas. 

Our  national  survey  of  district  Eisenhower  coordinators  indicates  that  although  the  primary 
content  focus  of  Eisenhower-assisted  activities  continues  to  be  on  mathematics  and  science,  some 
districts  have  begun  to  fund  professional  development  in  other  subject  areas  as  well.  We  asked 
district  coordinators  in  which  subject  areas  they  support  professional  development  using  Eisenhower 
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funds.  Exhibit  4.1  illustrates  the  percent  of  teachers  who  are  in  districts  that  fund  activities  in 
mathematics,  science,  language  arts,  and  several  other  subject  areas,  from  July  1 through  December 
1997.  As  the  Exhibit  shows,  nearly  all  teachers  are  in  districts  that  fund  professional  development 
activities  in  mathematics  (99  percent)  and  science  (99  percent).  Of  the  other  subject  areas,  44 
percent  of  teachers  are  in  districts  that  use  Eisenhower  funds  for  language  arts,  and  26  and  24  percent 
of  teachers  are  in  districts  that  use  Eisenhower  funds  for  professional  development  in  social  studies 
and  technology,  respectively.  In  fewer  cases,  teachers  are  in  districts  that  use  Eisenhower  funds  to 
provide  professional  development  in  the  arts  (five  percent),  special  education  (four  percent),  health 
and  physical  education  (two  percent),  and  other  areas  (12  percent)  (i.e.,  vocational  education,  home 
economics,  foreign  language,  and  activities  that  are  appropriate  for  multiple  subject  areas  such  as 
pedagogy  and  early  childhood  programs). 


EXHIBIT  4.1 

Percent  of  Teachers  in  Districts  Using  Eisenhower  Funds  to  Support  Professional 
Development  Activities,  by  Subject  Area  (n=3534) 


Subject  areas  supported  by  Eisenhower 


Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998,  reporting  on  the  1997-1998  school  year. 

How  to  read  this  exhibit:  The  first  bar  shows  that  99  percent  of  teachers  are  in  districts  that  use  Eisenhower  funds  to  support 
professional  development  in  mathematics.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each 
category. 


As  part  of  our  data  collection,  we  asked  Eisenhower  coordinators  to  provide  us  with  lists  of 
the  Eisenhower-assisted  professional  development  activities  that  they  offered  from  July  1 through 
December  1997,  and  the  number  of  teachers  and  other  staff  who  participated  in  each  activity.  An 
analysis  of  these  activity  lists  confirms  the  findings  from  the  district  coordinator  surveys,  shown  in 
Exhibit  4.1,  that  districts  support  Eisenhower  activities  in  subject  areas  outside  of  mathematics  and 
science. 


4 Of  our  total  sample  of  363  district  Eisenhower  coordinators,  10  districts  did  not  use  Eisenhower  funds  to  support 
professional  development  activities;  therefore  there  were  a total  of  353  district  coordinators  answering  questions 
about  Eisenhower  professional  development  activities. 
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Although  our  survey  data  and  information  from  the  activity  lists  indicates  that  districts  offer 
Eisenhower-assisted  professional  development  across  a range  of  subject  areas,  both  sources  of 
information  also  indicate  that  teachers  participate  mostly  in  mathematics  and  science-related 
activities.  Results  from  our  survey  of  Eisenhower  coordinators,  shown  in  Exhibit  4.2,  illustrate  how, 
for  each  district,  the  percent  of  teacher  participations  in  Eisenhower-assisted  professional 
development  activities  is  distributed  across  subject  areas;  the  mean  percent  of  participations  is  listed 
to  the  right  of  each  distribution.  It  is  evident  from  these  data  that  teacher  participation  is 
concentrated  on  professional  development  focused  in  mathematics  and  science.  On  average,  districts 
report  that  the  highest  percent  of  participations  is  in  mathematics  (44  percent)  and  the  second  highest 
is  in  science  (30  percent);  15  percent  of  participations  are  in  technology  alone,  or  technology  in 
combination  with  mathematics  and  science.  On  average,  only  1 1 percent  of  Eisenhower-assisted 
professional  development  participations  are  completely  outside  the  areas  of  mathematics,  science  and 
technology.5  None  of  these  patterns  differ  significantly  according  to  district  poverty  level  or  district 
size. 


EXHIBIT  4.2 

Percent  of  Participations  in  Eisenhower-assisted  Professional  Development 

Activities,  by  Subject  Area  (n=312) 
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Source : District  coordinator  lists  of  Eisenhower-assisted  activities  provided  in  the  district  from  July  1 through  December  31,  1997. 
How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  44  percent  of  participations  in  Eisenhower- 
assisted  activities  are  in  mathematics.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases, 
the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  districts  for  that  particular 
category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


Our  case-study  data  also  indicate  that  the  presence  of  Eisenhower  funding  in  subject  areas 
other  than  mathematics  and  science  remains  quite  modest.  Furthermore,  our  district  survey  and  case- 


5 We  did  not  collect  survey  data  on  the  amount  of  district  Eisenhower  funds  that  were  used  to  support  professional 
development  in  these  different  subject  areas;  therefore,  we  cannot  report  on  the  percentage  of  funds  that  are  spent  in 
subjects  other  than  mathematics  and  science. 
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study  data  both  suggest  that  Eisenhower  funding  is  essential  for  some  districts’  support  of 
professional  development  in  mathematics  and  science.  All  surveyed  districts  report  that  Eisenhower 
funds  are  important  to  their  ability  to  provide  professional  development  in  mathematics  and  science, 
and  45  percent  report  that  “most”  or  “all”  professional  development  in  mathematics  and  science  in 
their  districts  is  supported  with  Eisenhower  funds  (data  not  shown).6  Furthermore,  administrators  in 
some  case-study  districts  report  that  professional  development  in  mathematics  and  science  would  be 
neglected  without  Eisenhower  funds. 

For  example,  in  Northtown,  Connecticut,  all  Eisenhower  funds  go  to  professional 
development  in  mathematics  and  science,  because  the  Eisenhower  coordinator  feels  that  principals 
are  predisposed  to  spend  all  unspecified  professional  development  funds  on  reading.  The  coordinator 
suggests  that  it  is  important  to  set  aside  some  funds  to  ensure  the  availability  of  professional 
development  in  mathematics  and  science.  Similarly,  in  Maple  City,  Ohio,  about  80  percent  of  the 
professional  development  activities  conducted  by  the  districts’  mathematics  and  science  supervisors 
is  Eisenhower-assisted.  The  supervisors  say  that  if  it  were  not  for  Eisenhower  funds,  their 
professional  development  activities  in  mathematics  and  science  would  be  “in  a dismal  state.” 

In  general,  case  districts  use  their  Eisenhower  funds  for  professional  development  activities 
in  both  mathematics  and  science.  However,  some  case  districts  choose  to  emphasize  either 
mathematics  or  science  rather  than  divide  their  Eisenhower  funds  equally  between  the  two  subjects. 

In  Texas,  for  example,  the  state’s  testing  program  places  much  greater  emphasis  on  mathematics  than 
on  science.  Over  the  course  of  their  school  careers,  students  are  tested  five  times  in  mathematics,  but 
only  once  in  science.  The  district  coordinator  said  that  the  state’s  emphasis  on  mathematics  through 
its  assessment  system  led  district  administrators  to  focus  Eisenhower  funds  in  the  area  of 
mathematics,  rather  than  science.  Administrators  in  Rhinestone,  Texas,  state  that  the  majority  of 
Eisenhower  funds  are  spent  on  professional  development  in  mathematics,  because  the  district  wishes 
to  improve  scores  on  statewide  assessments  in  mathematics.  Thus,  perhaps  because  statewide 
assessments  generally  are  less  common  in  science  than  in  mathematics,  some  districts  may  be  more 
inclined  to  use  Eisenhower  funds  for  professional  development  in  mathematics  rather  than  in  science. 


6 The  Eisenhower  legislation  includes  a local  cost-sharing  requirement.  At  least  33  percent  of  the  cost  of  district 
Eisenhower  activities  must  be  borne  by  the  LEA  from  non-Title  II  sources.  These  sources  may  include  cash  or  in- 
kind  contributions,  and  may  come  from  a variety  of  sources,  including:  (1)  private,  non-federal  cash  contributions, 
and  (2)  release  time  for  teachers,  and  (3)  federal  funds.  The  stipulations  on  the  use  of  federal  funds  are  (1)  that  they 
are  used  consistently  with  Title  II  and  the  statute  under  which  the  funds  were  appropriated,  and  (2)  that  they  are  used 
to  benefit  students  and  teachers  who  otherwise  would  have  been  served  by  these  funds.  Furthermore,  the  SEA  may 
waive  this  requirement  if  the  LEA  is  unable  to  meet  the  requirement  due  to  economic  hardship,  and  that  the 
requirement  would  preclude  the  LEA’s  participation  in  the  program  (Section  2209).  Therefore,  the  Eisenhower 
legislation  appears  to  allow  Eisenhower  funds,  combined  with  other  federal  funds,  to  support  most  or  all 
mathematics  or  science  professional  development  activities  in  a district.  The  legislation  also  appears  to  allow 
Eisenhower  funds  alone  to  support  most  or  all  mathematics  and  science  professional  development  activities,  if  the 
LEA  has  obtained  a waiver  from  the  cost-sharing  requirement.  Also,  the  1994  legislation  no  longer  includes  a 
requirement  that  Eisenhower  funds  must  “supplement,  not  supplant”  other  funding  sources. 
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In  some  case  districts,  however,  Eisenhower  coordinators  report  that,  without  Eisenhower 
funds,  professional  development  in  science  would  not  exist.  For  example,  Eisenhower  funds  are 
primarily  spent  on  science  activities  in  East  City,  New  York,  where  they  play  a critical  role  in 
supporting  professional  development  in  science.  In  East  City,  the  twin  goals  of  literacy  and 
mathematics  dominate  every  school  effort.  Schools  spend  an  hour  every  school  day  on  sustained 
reading,  known  as  the  “golden  hour.”  The  hour  is  golden  because  it  is  sacrosanct,  and  cannot  be 
skipped  or  interrupted.  Mathematics  receives  similar  attention;  schools  spend  an  hour  every  school 
day  on  mathematics  instruction,  known  as  the  “silver  hour.”  Locally  funded  professional 
development  efforts  focus  on  literacy  and  mathematics  as  well. 

Science,  in  contrast,  is  viewed  as  a luxury  in  East  City,  according  to  district  officials. 
Although  districts  have  science  coordinators  who  organize  and  conduct  professional  development 
activities  for  the  district,  local  budget  cuts  have  been  aimed  at  these  positions.  According  to  district 
officials,  professional  development  in  mathematics  receives  support  from  many  sources;  science  now 
receives  reliable  support  from  only  one — Eisenhower  program  funds.  Therefore,  the  district  has 
decided  that  90  percent  of  Eisenhower  dollars  would  fund  professional  development  in  science. 

Similarly,  in  Riverside,  Washington,  Eisenhower  funds  have  been  instrumental  in  a 10-year 
effort  to  overhaul  the  elementary  science  curriculum,  according  to  interviews  with  district  officials. 
After  a period  of  planning,  the  overhaul  of  the  Riverside  science  curriculum  began  in  earnest  in  1990, 
when  many  of  the  district’s  teachers  attended  30-hour  classes  at  a local  university.  The  classes  were 
intended  to  help  teachers  understand  concepts  in  the  life,  earth,  and  physical  sciences.  Then,  in 
response  to  teachers’  reported  enthusiasm  for  the  courses,  some  teachers  took  advanced  classes  in 
these  concepts  over  the  next  few  years.  By  1993,  a core  group  of  about  30-50  teachers  was  trained 
and  ready  to  begin  creating  kits — curriculum  units  that  include  instructions  and  materials  to  help 
teach  them.  Currently,  there  are  three  or  four  kits  at  each  grade  level;  teachers  build  some  of  the  kits, 
and  others  are  purchased  from  outside  sources.  Now,  professional  development  in  Riverside 
involves  a “train  the  trainer”  model,  in  which  teachers  observe  classes  taught  by  those  teachers  who 
have  expertise  in  the  kits.  According  to  the  district  Eisenhower  coordinator,  Eisenhower  funds  have 
been  instrumental  in  developing  the  kit-based  science  curriculum  in  the  district,  and  in  fostering  a 
strategy  for  professional  development  in  science  that  is  more  coherent  than  the  district’s  approach  to 
professional  development  in  other  subjects. 

Some  of  our  case  districts  do  use  Eisenhower  funds  outside  of  mathematics  and  science. 

Data  from  these  districts  suggest  that  there  may  be  three  general  strategies  that  districts  take  to  using 
Eisenhower  funds  in  other  subject  areas.  One  strategy  is  to  hand  over  Eisenhower  resources  to  the 
district-level  staff  responsible  for  the  particular  subject  area  other  than  mathematics  or  science.  For 
example,  in  Commuteville,  Virginia,  the  district’s  language  arts  and  social  studies  specialists 
received  less  than  five  percent  of  the  district’s  Eisenhower  funds  to  provide  professional 
development  in  those  subjects.  The  district  had  plans  to  substantially  increase  Eisenhower  support  for 
social  studies  during  the  following  school  year,  to  over  20  percent  of  the  district’s  Eisenhower 
budget.  District  officials  say  that  this  would  begin  to  provide  a better  balance  in  professional 
development  opportunities,  because  social  studies  teachers  had  less  access  to  professional 
development  than  did  science  teachers. 
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A second  strategy  for  using  Eisenhower  funds  in  other  subjects  is  to  provide  professional 
development  activities  that  span  multiple  subject  areas.  In  Rhinestone,  Texas,  Eisenhower  funds 
support  an  activity  that  focuses  on  both  mathematics  and  language  arts.  For  the  activity,  a 
committee  of  teachers  developed  “Starstruck  Vocabulary,”  a set  of  vocabulary  words  pertaining  to 
mathematics  to  be  used  by  teachers  at  each  grade  level. 

A third  strategy  is  to  allow  schools,  in  districts  that  have  delegated  responsibility  for 
professional  development  to  schools,  to  elect  to  use  Eisenhower  funds  in  other  subject  areas.  In 
Richmond,  New  York,  schools  apply  for  Eisenhower  funds  from  the  district,  and  some  schools 
request  funds  to  support  professional  development  in  subject  areas  other  than  mathematics  or 
science. 

Summary:  District  Portfolios’  Emphasis  on  Mathematics  and  Science 

This  section  has  demonstrated  that  while  some  districts  use  Eisenhower  funds  to  support 
professional  development  in  areas  other  than  mathematics  and  science,  the  professional  development 
activities  that  teachers  participate  in  the  most  are  focused  on  mathematics  and  science.  In  1997-98, 
almost  23  percent  of  Eisenhower  funds  could  be  used  for  subjects  outside  of  mathematics  and 
science,  and  in  our  sample,  approximately  1 1 percent  of  teachers  are  in  districts  that  use  Eisenhower 
funds  for  non-mathematics  or  science  activities.  Our  results  may  indicate  fewer  non-mathematics  or 
science  activities  than  are  truly  being  implemented  because  we  did  not  collect  information  on  every 
non-mathematics/science  activity  that  a district  offered.  However,  if  expanding  the  use  of 
Eisenhower  funds  to  support  professional  development  in  subjects  other  than  mathematics  and 
science  is  intended  to  help  provide  more  flexibility  for  school  districts,  our  data  suggest  that  school 
districts  may  not  be  taking  advantage  of  that  flexibility.  In  our  previous  report  on  a series  of 
exploratory  case  studies  (Birman,  Reeve,  & Sattler,  1998)  we  discussed  a number  of  reasons  why  this 
might  be  the  case.  One  reason  is  that  mathematics  and  science  are  the  program’s  traditional  focus.  A 
second  reason  is  that  the  amount  of  program  funding  for  other  subjects,  in  the  absence  of  a waiver,  is 
insufficient  to  warrant  the  shift  in  focus.  Data  from  our  case  studies  suggest  a third  reason — that  in 
the  perspective  of  some  district  administrators,  there  would  be  little  professional  development  in 
mathematics,  and  even  less  in  science,  without  Eisenhower  funds. 

This  continuing  use  of  Eisenhower  funds  to  focus  on  mathematics  and  science  can  be 
considered  a positive  feature  of  the  program,  based  on  its  role  in  fostering  content-focused 
professional  development  opportunities.  In  the  last  chapter  we  highlighted  the  importance  of  a focus 
on  content  in  professional  development  in  changing  teacher  practice,  and  highlighted  literature 
indicating  that  professional  development  that  focuses  on  content  knowledge  appears  to  promote 
student  achievement  more  than  “generic”  professional  development  that  is  not  embedded  in  content. 
Having  a critical  mass  of  funding  available  in  a particular  content  area  over  a substantial  period  of 
time  helps  to  foster  professional  development  that  focuses  on  content  knowledge.  The  reliability  of 
Eisenhower  funding  supports  districts’  ability  to  engage  in  long-term  planning,  and  to  leverage  other 
funds  for  professional  development  (Birman,  Reeve,  & Sattler,  1998).  Thus,  the  continuity  and  focus 
provided  by  Eisenhower  funds  appears  to  have  enabled  some  districts  to  build  the  capacity  for 
designing  content-specific  professional  development  strategies. 

To  build  on  these  findings  about  the  subject-matter  focus  of  Eisenhower  supported  activities, 
we  now  turn  to  a description  of  district  portfolios  of  Eisenhower-assisted  activities. 


Structural  and  Core  Features  of  District  Eisenhower 

Portfolios 


Section  Findings 

♦ Nationwide,  Eisenhower  district  coordinators  report  that  almost  all  school  districts  use 
Eisenhower  funds  to  support  traditional  types  of  professional  development  (i.e.,  in- 
district workshops  and  institutes,  and  out-of-district  workshops  and  conferences ).  While 
many  districts  support  reform  types  of  professional  development  (i.e.,  study  groups, 
teacher  networks,  mentoring,  committees,  internships,  individual  research  projects), 
teacher  participation  in  these  reform  activities  is  generally  low. 

♦ Some  districts  rely  more  heavily  on  traditional  approaches  such  as  workshops,  than  on 
other  approaches,  but  districts  vary  substantially  in  the  characteristics  of  the  workshops 
that  they  support.  Some  districts  support  workshops  of  relatively  long  duration  that 
emphasize  collective  participation  and  foster  opportunities  for  active  learning.  Similarly, 
reform  types  of  professional  development  vary  in  their  duration;  many  districts  offer 
reform  types  of  activities  that  span  less  than  one  month. 

♦ Large  districts  are  more  likely  than  others  to  have  a higher  percentage  of  participations 
in  reform  activities,  and  to  have  activities  that  span  a longer  period  of  time,  offer  more 
active  learning  strategies,  and  more  collective  participation. 

♦ Almost  all  districts  that  offer  traditional  types  of  activities  use  Eisenhower  funds  to 
support  at  least  some  of  them;  however,  many  districts  that  offer  reform  types  of  activities 
do  not  use  Eisenhower  funds  to  support  them. 

The  Title  II  legislation  stipulates  that  Eisenhower-assisted  activities  should  be  “sustained  and 
intensive,”  “ongoing,”  and  of  “high  quality,”  but  it  does  not  provide  detailed  guidance  regarding  the 
characteristics  of  activities  that  districts  should  fund. 

In  the  last  chapter,  we  examined  teacher  reports  of  Eisenhower-assisted  activities  along  a 
number  of  dimensions.  We  examined  the  structural  features  of  professional  development  activities — 
traditional  vs.  reform  types,  duration  and  collective  participation — and  core  features — content 
emphasis,  opportunities  for  active  learning,  and  coherence.  We  found  that  all  of  these  dimensions 
were  associated,  either  directly  or  indirectly,  with  enhanced  teachers’  knowledge  and  skills  and 
changes  in  teaching  practice.  We  continue  to  use  this  framework  in  this  chapter,  but  here  we  focus  on 
how  district  portfolios  of  activities  offer  opportunities  for  professional  development  that  have  the 
features  we  found  to  be  associated  with  teachers’  learning  and  change  in  teaching  practice. 

As  discussed  in  Chapter  3,  research  on  teachers’  professional  development  suggests  that 
high-quality  professional  development  is  characterized  by  activities  that  allow  teachers  to  focus  in 
depth  on  the  content  that  they  are  trying  to  master  and  on  how  children  learn  that  content  (Cohen  & 
Hill,  1998;  Fennema  et  al.,  1996;  Hiebert,  1999;  Kennedy,  1998;  Sparks  & Loucks-Horsley,  1989). 
High  quality  also  is  associated  with  activities  that  are  long  in  duration,  allow  teachers  an  opportunity, 
to  practice  and  reflect  upon  their  teaching,  and  are  embedded  in  the  ongoing  work  of  the  school. 
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Some  traditional  types  of  professional  development,  such  as  workshops  and  conferences,  are  thought 
to  be  less  likely  to  have  these  characteristics  (Little,  1993).  One  reason  is  that  traditional  approaches 
often  are  isolated  from  teaching  practice  and  characterized  by  “one-shot”  workshops  at  which 
teachers  listen  passively  to  “experts”  and  learn  about  topics  that  are  not  essential  to  teaching 
(National  Foundation  for  the  Improvement  of  Education,  1996). 

Although  teachers  sometimes  value  such  workshops  or  conferences  because  they  increase 
their  awareness  or  rejuvenate  their  interest  (Knapp,  Zucker,  Adelman,  & St.  John,  1991),  reform 
types  of  professional  development  such  as  study  groups,  teacher  networks,  mentoring,  committees  or 
task  forces,  internships,  and  individual  research  projects  appear  to  have  the  potential  for  a stronger 
impact  on  teaching  practice  (Darling-Hammond,  1995,  1997b;  Hargreaves  & Fullan,  1992;  Little, 
1993;  Richardson,  1994;  Sparks  & Loucks-Horsley,  1989;  Stiles,  Loucks-Horsley,  & Hewson,  1996). 
This  is  because  reform  types  of  professional  development  seem  more  likely  than  traditional  types  to 
offer  opportunity  for  reflection,  in-depth  study,  and  prolonged  focus,  and  thus  are  more  likely  to  be 
associated  with  changes  in  teacher  practice  and  improvements  in  student  achievement  (Sparks  & 
Loucks-Horsley,  1989). 

In  Chapter  3,  we  found  that  these  reform  activities  offer  more  opportunities  for  active 
learning  and  other  core  features  that  reflect  high-quality  professional  development  than  traditional 
types  of  activities  do.  Our  findings  in  Chapter  3 also  indicated  that  reform  types  of  professional 
development  are  more  likely  to  be  of  substantial  duration  and  allow  collective  participation,  which  in 
turn  are  related  to  high-quality  core  features  such  as  active  learning  opportunities  and  a focus  on 
content  knowledge.  However,  according  to  the  teachers  in  our  National  Profile,  to  some  extent 
traditional  types  of  professional  development  also  offer  some  of  these  high-quality  core 
characteristics,  depending  upon  their  duration  and  other  structural  features. 

Consequently,  in  this  chapter,  we  not  only  describe  the  district  provision  and  participation 
rates  for  traditional  and  reform  types  of  professional  development,  but  we  provide  information  about 
the  average  number  of  contact  hours  and  span  (i.e.,  in  days,  weeks,  months,  or  years)  of  the  activities 
that  districts  support,  and  the  different  types  of  learning  methods  used  in  the  activities.  In  examining 
the  quality  of  professional  development  activities  in  this  section,  there  is  one  aspect  of  these  activities 
that  we  do  not  focus  on  directly — their  content.  We  were  not  able  to  collect  detailed  information 
about  activity  content  from  district  coordinators  because  of  the  wide  range  of  professional 
development  activities  that  districts  provided.  Therefore,  we  focus  our  discussion  on  the  structural 
aspects  of  professional  development  activities  that  appear  to  be  related  to  desirable  core  features  and 
positive  teacher  outcomes — whether  they  are  traditional  or  reform,  their  average  duration  and 
whether  they  offer  opportunities  for  collective  participation.  We  also  present  information  for  one 
core  feature  for  which  we  do  have  district-level  information:  opportunities  for  active  learning. 

The  first  set  of  analyses  in  this  section  focuses  on  the  districts’  use  of  traditional  types  of 
professional  development.  We  then  discuss  district  provision  of  reform  approaches,  followed  by  an 
analysis  of  the  district’s  overall  strategy  of  professional  development,  which  compares  Eisenhower- 
assisted  activities  in  the  context  of  the  district’s  complete  portfolio  of  professional  development 
activities. 

Traditional  Types  of  Professional  Development 

During  our  telephone  interviews  with  district  Eisenhower  coordinators,  we  asked  whether 
their  district  supported  in-district  workshops  or  institutes  from  July  1 through  December  1997.  We 
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then  asked  whether  any  of  the  workshops  or  institutes  were  supported,  at  least  in  part,  with 
Eisenhower  funds.  We  asked  the  same  questions  about  out-of-district  workshops  and  conferences. 
We  also  asked  about  a third  traditional  type  of  professional  development — college  courses.  But  we 
do  not  report  on  this  type  because  the  percent  of  teacher  participations  in  courses  supported  with 
Eisenhower  funds  is  negligible.  Essentially,  Eisenhower  funds  are  very  rarely  used  to  support 
attendance  at  college  courses,  according  to  district  coordinators.  As  we  would  expect,  our  findings 
from  our  survey  of  project  directors  at  Institutions  of  Higher  Education  (IHEs)  indicate  that  IHEs 
often  use  Eisenhower  funds  to  support  college  courses;  consequently,  this  type  of  professional 
development  is  discussed  in  Chapter  6,  where  we  present  an  analysis  of  our  data  from  IHEs. 

On  our  survey,  district  Eisenhower  coordinators  indicated  how  many  participants  attended 
Eisenhower- assisted  workshops,  institutes,  or  conferences,  counting  participants  more  than  once  if 
they  attended  multiple  activities.  Exhibit  4.3  demonstrates  that,  by  adding  the  percent  of 
participations  in  in-district  workshops  and  institutes  and  out-of-district  workshops  and  conferences, 
on  average,  over  three-quarters  of  total  participations  in  Eisenhower-assisted  activities  (77  percent) 
are  in  these  two  types  of  traditional  activities,  with  substantially  more  participations  in  in-district 
workshops  and  institutes  than  out-of-district  workshops  and  conferences  (56  percent  compared  to  21 
percent,  respectively). 


EXHIBIT  4.3 

Percent  of  Participations  in  Traditional  Types  of  Eisenhower-assisted  Activities 

(n=353) 


Types  of  "traditional"  activities 


Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  56  percent  of  participations  in 
Eisenhower- assisted  activities  are  in  in-district  workshops  and  institutes.  Each  dot  represents  one  district.  As  the  number  of 
districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents 
the  distribution  of  districts  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 
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While  most  districts  use  Eisenhower  resources  to  fund  traditional  approaches  to  professional 
development,  districts  differ  dramatically  in  the  extent  of  their  reliance  on  these  approaches  for 
providing  professional  development  to  teachers.  Exhibit  4.3  indicates  that,  on  average,  districts  report 
that  56  percent  of  Eisenhower-assisted  participations  are  in  in-district  workshops  and  institutes. 
However,  in  some  districts,  no  teachers  who  participate  in  Eisenhower-assisted  activities  attend  in- 
district workshops  and  institutes,  while  in  other  districts  all  teachers  who  participate  in  Eisenhower- 
assisted  activities  attend  in-district  workshops  and  institutes.  The  same  variation  in  distribution  is 
true  of  out-of-district  workshops  and  conferences. 

Our  data  suggest  that  most  districts  rely  heavily  on  traditional  forms  of  professional 
development  in  their  use  of  Eisenhower  funds.  If  traditional  types  of  professional  development  are 
less  likely  to  be  “sustained  and  intensive,”  then  most  districts  are  not  using  Eisenhower  funds  in  line 
with  the  intent  of  the  provisions  of  the  1994  reauthorization.  However,  it  is  clear  from  the  last 
chapter  that  traditional  types  of  professional  development  activities  can  vary  in  their  structure  and 
substance.  Further,  districts  may  have  more  participations  in  traditional  than  in  reform  types  of 
activities,  but  the  extent  to  which  this  represents  disproportionate  spending  is  not  clear,  given  that 
reform  activities  may  be  more  expensive  per  participant  than  traditional  activities. 

Next  we  delve  more  deeply  into  districts’  support  of  traditional  types  of  professional 
development  activities  by  examining  structural  and  core  features  of  these  activities — specifically  the 
duration  and  extent  of  opportunities  for  collective  participation,  and  opportunities  for  teachers  to 
engage  in  active  learning  in  traditional  professional  development  activities.  These  analyses  help  to 
demonstrate  whether  and  how  districts  differ  in  the  quality  of  traditional  types  of  professional 
development  activities  that  they  support  with  Eisenhower  funds. 

The  Structural  and  Core  Features  of  Traditional  Activities 

As  discussed  in  Chapter  3,  certain  structural  features  are  associated  with  desirable  core 
features  of  professional  development  activities,  which  in  turn  are  associated  with  improved  teacher 
outcomes.  Specifically,  the  duration  of  the  activity,  which  includes  both  the  number  of  contact  hours 
and  the  time  span  over  which  the  activity  extends  (i.e.,  the  number  of  days,  weeks,  months,  or  years 
across  which  the  activity  is  spread),  as  well  as  the  combination  and  number  of  types  of  active 
learning  used  in  the  activity,  are  indications  of  high-quality  professional  development  (Cohen  & 

Hill,  1998;  Sparks  & Loucks-Horsley,  1989). 

The  fact  that  many  school  districts  rely  heavily  on  in-district  workshops  in  their  Eisenhower- 
assisted  activities  could  be  cause  for  concern,  since  such  workshops  often  are  assumed  to  be  short 
experiences  that  are  disconnected  from  ongoing  teacher  practice  (National  Foundation  for  the 
Improvement  of  Education,  1996).  The  Eisenhower  legislation’s  language  that  professional 
development  activities  be  “sustained  and  intensive”  implies  that  high-quality  professional 
development  should  involve  a relatively  large  investment  of  time  and  should  be  spread  over  a long 
period  of  time.  Sustained,  intensive  professional  development  activities  would  allow  teachers 
opportunities  to  absorb  complex  content,  practice  new  techniques  in  their  own  classrooms,  and 
discuss  their  experiences  with  other  teachers.  Short,  “one-shot”  workshops  would  not  generally 
provide  teachers  such  opportunities. 


Duration.  Data  from  our  national  survey  of  district  Eisenhower  coordinators  suggest, 
however,  that  in-district  workshops  are  not  always  “one-shot”  events.7  We  asked  district 
coordinators  to  report  on  the  percentages  of  in-district  workshops  and  institutes  that  lasted  varying 
numbers  of  hours,  including  follow-up  events.  As  shown  in  Exhibit  4.4a,  coordinators  indicated 
what  percent  of  Eisenhower-assisted  workshops  or  institutes  lasted  less  than  four  hours,  between  four 
and  eight  hours,  between  nine  and  40  hours,  and  more  than  40  hours. 

EXHIBIT  4.4a 


Percent  of  Eisenhower-assisted  In-district  Workshops  and  Institutes, 

by  Contact  Hours  (n=3148) 


Number  of  hours  of  in-district  workshops  or  institutes 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  18  percent  of  Eisenhower-assisted  in-district  workshops  and  institutes  last 
for  less  than  four  hours.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the  dots  form  a 
horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  districts  for  that  particular  category.  The  number  to  the 
right  of  the  distribution  is  the  mean. 


The  range  observed  across  districts  on  these  measures  suggests  that  districts  vary  greatly  in 
terms  of  the  number  of  hours  of  their  workshops  or  institutes.  For  all  four  categories  of  total  hours, 
the  minimum  reported  was  zero  percent  and  the  maximum  reported  was  100  percent.  In  other  words, 
there  are  districts  in  which  all  in-district  workshops  and  institutes  last  less  than  four  hours,  and  there 
are  districts  in  which  no  workshops  or  institutes  last  less  than  four  hours;  and  this  pattern  applies  to 
the  other  three  time  categories  as  well. 


7 We  did  not  ask  district  coordinators  about  the  duration  of  out-of-district  workshops  or  conferences.  Based  on  data 
collected  during  pilot-testing  of  the  survey  instrument,  out-of-district  workshops  and  conferences  all  were  assumed  to  las 
less  than  one  month.  Therefore  the  analyses  on  the  quality  of  traditional  professional  development  activities  focuses  on 
in-district  workshops  and  institutes. 

8 Of  our  total  sample  of  363  district  Eisenhower  coordinators,  314  provided  Eisenhower-assisted  in-district 
workshops  and  institutes;  therefore  all  analyses  that  pertain  to  in-district  workshops  and  institutes  have  a sample  of 
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Given  this  range  of  findings,  it  is  clear  that  some  districts  support  workshops  that  seem  to 
meet  the  high-quality  professional  development  criterion  of  “sustained  and  intensive.”  It  is  also  clear 
that  most  districts  do  not. 

In  addition  to  asking  about  the  number  of  contact  hours  of  in-district  workshops  and 
institutes,  we  asked  Eisenhower  coordinators  to  report  the  percent  of  their  Eisenhower-assisted 
workshops  or  institutes,  including  follow-up  activities,  that  were  spread  across  different  time  periods. 
The  options  were  one  day,  two  to  seven  days,  eight  days  to  one  month,  one  month  to  a year,  and 
more  than  a year. 

Exhibit  4.4b  shows  that  district  coordinators  report  that  many  in-district  workshops  and 
institutes  span  a relatively  short  period  of  time.  Some  districts,  however,  support  workshops  and 
institutes  that  span  a relatively  long  period  of  time.  These  findings  demonstrate  that,  as  with  the 
number  of  hours,  span  across  time  in  days,  weeks  and  months  varies  considerably  from  district  to 
district.  As  Exhibit  4.4b  shows,  districts  that  support  in-district  workshops  and  institutes  report  that 
more  than  three-quarters  of  their  workshops  extend  over  less  than  one  month. 

EXHIBIT  4.4b 

Percent  of  Eisenhower-assisted  In-district  Workshops  and  Institutes,  by  Time  Span 

(n=314) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  38  percent  of  in-district  workshops  or  institutes  last  for  one  day 
only.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line 
that  increases  in  length.  Each  distribution  represents  the  distribution  of  districts  for  that  particular  category.  The  number  to  the  right 
of  the  distribution  is  the  mean. 


Although  we  do  not  have  information  about  the  quality  of  the  learning  that  takes  place  in 
these  activities,  they  would  seem  to  lack  the  “sustained”  characteristic  associated  with  high-quality 
professional  development.  At  the  same  time,  almost  a quarter  of  Eisenhower  in-district  workshops 
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and  institutes  span  over  a month  or  more.  Given  the  common  perception  of  workshops  as  very  short 
learning  opportunities,  this  is  perhaps  a surprising  finding.  It  should  be  noted,  that  in  the  early  1990s, 
districts  were  providing  Eisenhower-assisted  activities  of  shorter  duration  than  they  are  presently — a 
median  of  six  hours  compared  to  the  current  median  of  15  hours  (Knapp  et  al.,  1991).  Thus,  since 
the  1994  reauthorization  of  ESEA  and  its  emphasis  on  “sustained  and  intensive”  activities,  districts 
have  increased  the  duration  of  the  professional  development  activities  that  they  provide. 

Collective  participation.  In  addition  to  being  too  short,  both  in  number  of  contact  hours  and 
time  span,  traditional  professional  development  also  has  been  criticized  as  being  geared  too  little 
toward  developing  approaches  that  allow  teachers  to  work  together  over  time  and  exchange 
information  to  enhance  their  teaching  of  a subject  area  (Little,  1993).  While  high-quality 
professional  development  should  meet  the  learning  needs  of  individual  teachers,  systemic  reform 
emphasizes  that  teachers  in  the  same  school  or  who  teach  the  same  subject  ought  to  have  consistent 
knowledge  bases  and  approaches  to  teaching  and  learning  (O’Day  & Smith,  1993).  Collective 
participation  by  whole  schools,  a critical  mass  of  teachers,  or  particular  groups  of  teachers  in  a school 
allows  teachers  to  share  effective  practices  and  reinforces  new  ideas  and  methods  (Newmann  et  al., 
1996);  it  also  provides  opportunities  for  teachers  to  discuss  issues,  and  offers  them  a basis  for 
comparing  and  choosing  alternative  practices  (Ball,  1996).  Our  study  of  teachers,  reported  in 
Chapter  3,  suggests  that  collective  participation  in  professional  development,  which  offers  the 
opportunity  to  foster  a common  body  of  knowledge  among  teachers,  is  associated  with  teachers’  self- 
reported  enhanced  knowledge  and  skills.  Further,  the  authorizing  legislation  acknowledges  its 
importance  by  specifying  that  districts  can  use  Eisenhower  funds  to  support  professional 
development  geared  not  just  toward  individual  teachers’  needs  but  also  toward  the  needs  of  groups  of 
teachers  (Section  2210(b)(3)(A)). 

To  measure  this  idea  of  collective  participation,  or  the  extent  to  which  districts  provide 
opportunities  for  in-district  workshops  designed  for  groups  of  teachers  or  all  teachers  in  a school,  we 
asked  Eisenhower  coordinators  for  whom  they  designed  their  workshops.  Coordinators  indicated 
whether  workshops  and  institutes  were  specifically  designed  for:  1)  all  teachers  in  department  or 
grade-level  groupings  and/or  2)  all  teachers  in  a school  or  set  of  schools,  as  opposed  to  being 
designed  for  teachers  as  individuals  or  teachers  as  representatives  of  their  departments,  grade  level, 
or  school. 

Exhibit  4.5a  shows  that  almost  three-quarters  (74  percent)  of  teachers  are  in  districts  that 
design  in-district  workshops  and  institutes  for  all  teachers  in  a department  or  grade,  while  58  percent 
of  teachers  are  in  districts  that  design  workshops  for  all  teachers  in  a school  or  set  of  schools. 
However,  while  most  districts  report  that  they  use  Eisenhower  funds  to  support  workshops  and 
institutes  that  provide  opportunities  for  collective  participation,  we  do  not  have  data  about  how 
frequently  districts  support  workshops  and  institutes  that  offer  such  opportunities. 
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EXHIBIT  4.5a 

Percent  of  Teachers  in  Districts  in  Which  Eisenhower-assisted  In-district  Workshops 
and  Institutes  Include  Collective  Participation  (n=314) 


Groups  for  whom  district  designed  in-district 
workshops  and  institutes 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  74  percent  of  teachers  are  in  districts  that  design  in-district  workshops  and  institutes 
for  all  teachers  in  a department  or  grade  level.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each 
category.  


We  combined  and  averaged  responses  to  these  two  questions  to  create  an  index  of  the  extent 
to  which  districts  provide  opportunities  for  collective  participation  in  professional  development 
activities  that  go  beyond  the  needs  of  individual  teachers;  the  scale  is  from  zero  to  one,  where  zero 
indicates  no  opportunities  for  collective  participation,  .5  indicates  one  opportunity  for  collective 
participation,  and  one  indicates  opportunities  for  both  types  of  collective  participation.  Exhibit  4.5b 
illustrates  that  while  collective  participation  opportunities  do  not  vary  by  the  poverty  level  of  the 
district,  large  and  medium  districts  are  significantly  more  likely  to  design  in-district  workshops  to 
offer  collective  participation  than  are  small  districts.  Smaller  districts  may  not  have  enough  teachers 
across  grade  levels  or  departments  to  justify  designing  activities  for  them.  Also,  compared  to  small 
districts,  large  districts  may  have  more  funding  sources  for  professional  development  and  therefore 
have  more  resources  to  serve  whole  schools  or  large  groups  of  teachers. 


EXHIBIT  4.5b 


Collective  Participation  in  Eisenhower-assisted  In-district  Workshops  and  Institutes, 
Overall  and  by  District  Poverty  and  District  Size  (n=314) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  a collective  participation  score  of  .66  where  zero 
indicates  no  opportunities  for  collective  participation  in  in-district  workshops  and  institutes  and  one  indicates  that  the  district  offers  both 
types  of  collective  participation  in  in-district  workshops  and  institutes.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one 
data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  the 
number  of  opportunities  for  collective  participation  that  districts  offer  in  their  in-district  workshops  and  institutes.  The  number  to  the  right 
of  the  distribution  is  the  mean.  


Opportunities  for  active  learning.  Workshops  vary  not  only  in  their  duration  and 
opportunities  for  collective  participation,  but  also  in  the  types  of  opportunities  they  provide  for 
teachers  to  practice  what  they  have  learned.  Active  learning  is  an  important  dimension  of  the  quality 
of  professional  development,  as  we  demonstrated  in  Chapter  3.  While  the  term  “workshop”  may 
evoke  the  image  of  a relatively  traditional  learning  format,  workshops  can  vary  along  this  dimension. 
“At  their  best,  [workshops]  provide  adult  learners  with  important  and  relevant  new  knowledge  and 
opportunities  to  try  new  ideas,  practice  new  behaviors,  and  interact  with  others  as  they  learn” 
(Loucks-Horsley,  Hewson,  Love,  & Stiles,  1998,  p.  88).  For  any  type  of  activity,  active  learning 
opportunities  are  critical  to  allow  teachers  to  reflect,  discuss,  and  practice  new  ideas  and  methods 
(Carey  & Frechtling,  1997;  Carpenter  et  al.,  1989;  Darling-Hammond,  1997b;  Lieberman,  1996; 
Schifter,  1996).  Although  the  extent  of  in-depth,  reflective  learning  depends  to  a large  extent  on  the 
content  of  the  curricula,  the  use  of  active  learning  strategies  provides  one  measure  of  the  opportunity 
for  such  conceptual  learning. 

To  find  out  about  the  opportunities  for  active  learning  that  in-district  workshops  and 
institutes  offer  to  participants,  we  asked  district  Eisenhower  coordinators  which  of  the  following 
strategies  were  used  regularly  in  Eisenhower-assisted  workshops  or  institutes  to  help  teachers 
implement  new  skills: 
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♦ observe  professional  development  providers  demonstrating  or  modeling  skills, 

♦ Observe  other  teachers  teaching, 

♦ practice  tinder  simulated  conditions  with  feedback, 

♦ meet  in  groups  to  discuss  problems  in  implementation,  and 

♦ be  observed  teaching  in  his  or  her  own  classroom. 

Exhibit  4.6  shows  the  relative  frequency  with  which  districts  report  using  each  of  these 
strategies  in  their  in-district  workshops  and  institutes.  As  the  exhibit  illustrates,  virtually  all  districts 
report  that  they  use  Eisenhower  funds  to  support  workshops  in  which  teachers  have  to  meet  in  groups 
to  discuss  problems  with  the  implementation  of  new  practices  and  to  observe  professional 
development  providers  demonstrating  skills. 

EXHIBIT  4.6 

Percent  of  Teachers  in  Districts  That  Provide  Each  of  Five  Types  of  Opportunities  for 
Active  Learning  in  Eisenhower-assisted  In-district  Workshops  and  Institutes  (n=314) 
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Source.  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  96  percent  of  teachers  are  in  districts  that  offer  in-district  workshops  or  institutes 
that  provide  opportunities  for  participants  to  observe  the  workshop  leader  demonstrating  or  modeling  skills.  Each  bar  and  the  number 
on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 


As  Exhibit  4.6  shows,  fewer  districts  use  Eisenhower  funds  to  provide  the  other  three  types 
of  active  learning  opportunities.  Almost  three-quarters  (74  percent)  of  teachers  are  in  districts  in 
which  district  coordinators  report  that  in-district  workshops  and  institutes  allow  participants  to 
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observe  others,  and  almost  70  percent  are  in  districts  that  report  that  the  workshops  and  institutes 
provide  opportunities  to  simulate  practice.  However,  only  about  half  of  teachers  are  in  districts  (49 
percent)  that  support  workshops  and  institutes  that  include  opportunities  to  have  teachers  observed 
teaching  in  their  own  classrooms,  perhaps  the  strategy  most  likely  to  relate  to  change  in  teacher 
practice.  Since  this  part  of  the  survey  specifically  asked  coordinators  to  indicate  which  activities  were 
conducted  “regularly”  in  workshops,  these  data  provide  an  approximate  measure  of  the  frequency 
and/or  prevalence  of  these  opportunities  for  active  learning  in  the  workshops,  from  the  perspective  of 
district  Eisenhower  coordinators. 

To  examine  how  varied  a district’s  workshop  learning  methods  are,  we  developed  a scale  that 
represents  each  district’s  provision  of  active  learning  opportunities  for  teachers  in  Eisenhower- 
assisted  in-district  workshops  and  institutes.  The  scale  is  a composite  that  combines  the  five 
strategies  for  active  learning  (i.e.,  observe  leaders  demonstrate,  observe  other  teachers,  simulate 
practice,  discuss  problems  with  the  implementation  of  new  techniques,  and  have  their  own  practice 
observed).9 

Exhibit  4.7  shows  on  average  how  many  of  these  five  relative  types  of  opportunities  for 
active  learning  districts  report  using  during  in-district  workshops  and  institutes.  By  and  large,  district 
coordinators  report  that  Eisenhower-assisted  workshops  and  institutes  provide  teachers  with  four  of 
the  five  types  of  learning  opportunities. 


EXHIBIT  4.7 


Number  of  Types  of  Opportunities  for  Active  Learning  in  Eisenhower-assisted  In- 
district Workshops  and  Institutes,  Overall  and  by  District  Poverty  and  District  Size 

(n=314) 
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Size 

Significant  Pairwise  Contrasts 

Small  vs.  Medium,  Small  vs.  Large,  Small  vs.  Consortium,  Medium  vs.  Large 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  districts  report  that  they  offer  3.9  types  of  opportunities  for 
active  learning  in  their  in-district  workshops  and  institutes.  The  number  of  types  of  opportunities  for  active  learning  differs 
significantly  by  district  size,  but  not  by  district  poverty  level.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data 
point  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  the  number  of 
opportunities  for  active  learning  for  that  particular  category.  The  number  on  the  distribution  is  the  mean. 


9 See  Appendix  D for  more  details  about  the  scale’s  composition  and  reliability. 
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However,  Exhibit  4.7  also  indicates  that  the  larger  the  district,  the  more  likely  the  district  is 
to  provide  Eisenhower-assisted  activities  that  are  characterized  by  multiple  opportunities  for  active 
learning.  Small  districts  provide  significantly  fewer  opportunities  for  active  learning  than  medium  or 
large  districts  or  consortia,  and  medium  districts  provide  fewer  active  learning  opportunities  than 
large  districts.  Here  consortia  outperform  small  districts,  and,  as  with  collective  participation,  large 
districts  outperform  small  districts. 

Case-study  Examples  of  District  Support  for  Traditional  Types  of  Professional 
Development 

Consistent  with  our  data  from  teachers  reported  in  Chapter  3,  the  data  from  district 
coordinators  presented  so  far  indicate  that  most  school  districts  continue  to  rely  heavily  on  traditional 
types  of  professional  development.  For  the  most  part,  these  activities  last  less  than  eight  hours  and 
extend  over  a span  no  greater  than  a week.  These  results  appear  to  indicate  that  most  districts  have  a 
way  to  go  in  order  to  meet  the  intent  of  the  1994  reauthorization  to  provide  “sustained,  intensive” 
professional  development.  However,  we  do  not  know  the  proportion  of  Eisenhower  funds  that 
districts  spend  on  traditional  types  of  activities;  since  reform  approaches  to  professional  development 
may  be  more  expensive  per  participant  than  traditional  approaches,  a district  may  have  more 
participations  in  traditional  activities,  but  spend  a higher  proportion  of  funds  on  reform  activities. 

Further,  while  most  teacher  participations  are  in  traditional  types  of  professional  development 
with  short  durations,  the  data  from  district  Eisenhower  coordinators  also  indicate  that  some  districts 
are  able  to  use  their  Eisenhower  funds  for  traditional  professional  development  activities  that  have 
features  of  high  quality. 

Our  case-study  data  illustrate  how  traditional  types  of  professional  development  can  have 
characteristics  of  high  quality.  In  Rainforest,  Washington,  a small,  rural  district,  all  Eisenhower 
funding  goes  to  support  one  major  professional  development  activity  each  year:  a science  institute. 
The  institute  employs  a fairly  traditional  format — five  days  of  classes  during  the  summer.  However, 
it  extends  over  the  subsequent  school  year  through  in-class  observations,  modeling,  and  coaching  of 
teachers  by  the  district’s  science  coordinator,  a well-respected  leader  in  her  field.  The  institute  also 
is  characterized  by  collective  participation,  since  all  science  teachers  in  the  district — that  is,  all 
elementary  teachers  as  well  as  high  school  science  teachers — are  expected  to  participate.  Thus  all 
teachers  who  teach  science  in  the  district’s  schools  will  have  shared  the  same  learning  experience. 
This  is  an  example  of  how  one  district  is  able  to  use  Eisenhower  funds  for  professional  development 
that  has  a traditional  format,  but  has  features  of  high-quality  professional  development. 

East  City,  New  York,  is  another  example  of  a district  that  invests  Eisenhower  funds  in 
professional  development  activities  that  have  a traditional  format,  but  with  features  of  high  quality. 
The  primary  activities  supported  by  Eisenhower  funds  in  this  large,  urban  district  are  grounded  in  the 
district’s  “partnerships”  with  cultural  and  science-related  institutions.  Institutions  such  as  the  local 
botanical  gardens  and  the  local  zoo  have  relationships  with  the  district  to  support  science  education, 
and  the  associated  professional  development,  for  teachers  in  selected  schools.  The  teachers  in  the 
schools  participate  in  these  activities  only  if  the  school  itself  has  demonstrated  sufficient 
administrative  support  for  the  partnership,  illustrating  how  the  activities  are  linked  to  collective 
participation. 

The  partnership  with  the  local  zoo  is  characteristic  of  the  district’s  approach.  The  zoo 
provides  multi-session  summer  workshops  lasting  21  to  36  hours,  a traditional  type  of  professional 
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development  that  extends  over  a much  longer-than-average  time  period.  This  workshop  introduces 
teachers  in  selected  schools  to  a full  curriculum  related  to  the  zoo’s  exhibits  and  activities;  thus,  the 
workshops  focus  on  particular  schools  and  emphasize  collective  participation.  Furthermore,  the 
summer  workshop  is  followed  up  throughout  the  school  year  by  visits  from  zoo  staff,  and  access  to 
teaching  materials  and  programs  for  students. 

The  examples  of  Rainforest  and  East  City  illustrate  how  two  districts,  one  a small  rural 
community,  and  the  other  a large  urban  area,  have  used  substantial  proportions  of  their  Eisenhower 
funds  to  support  traditional  types  of  professional  development — workshops  and  institutes — that  have 
features  of  high-quality  professional  development.  In  particular,  the  activities  extend  over  a longer- 
than-typical  time  period,  are  geared  toward  collective  participation,  and,  in  Rainforest,  build  in 
opportunities  for  observation  of  teaching  practice  and  coaching.  While  our  survey  data  indicate  that 
districts  that  use  most  of  their  funding  for  these  types  of  activities  are  not  the  norm,  these  cases  also 
illustrate  that  such  uses  of  Eisenhower  funds  can  occur. 

Reform  Types  of  Professional  Development 

We  now  turn  from  traditional  types  of  professional  development  to  discuss  districts’  use  of 
Eisenhower  funds  to  support  reform  types  of  professional  development.  As  discussed  earlier  in  this 
report,  some  types  of  professional  development  have  features  that  make  them  particularly  compatible 
with  systemic  reform,  because  they  appear  to  afford  teachers  the  opportunity  to  learn  content 
knowledge  in  greater  depth  than  more  traditional  types  of  professional  development  and  give 
teachers  more  of  an  opportunity  to  reflect  on  what  they  have  learned.  These  types  of  professional 
development  also  may  afford  teachers  the  opportunity  to  work  together  and  learn  from  one  another  so 
that  they  have  common  understandings  about  content,  and  teaching  and  learning  (O’Day  & Smith, 
1993).  Finally,  these  reform  types  of  professional  development  also  might  be  more  embedded  in 
ongoing  activities,  so  they  are  more  integrated  into  the  daily  life  of  schools  than  more  traditional 
types  of  professional  development  (Elmore,  1996;  Little,  1993). 

In  this  part  of  the  chapter  we  examine  how  districts  differ  in  their  use  of  Eisenhower  funds  to 
support  the  following  reform  types  of  professional  development,  as  defined  in  Chapter  3:  teacher 
study  groups,  teacher  collaboratives  or  networks,  mentoring,  committees  or  task  forces,  internships, 
and  individual  research  projects.  We  asked  about  an  additional  type  of  professional 
development — teacher  resource  rooms.  But  this  reform  type  of  professional  development  is  not 
discussed  in  the  analyses  in  this  chapter  because  the  percent  of  teachers  who  participate  in  such 
resource  rooms  is  very  small  (i.e.,  .04  percent).  Our  data  show  that  teacher  resource  rooms  are  the 
least  likely  professional  development  activity  to  be  offered  to  teachers  as  part  of  a district’s 
Eisenhower-assisted  professional  development  portfolio.  Therefore,  we  focus  on  the  other  six  types 
of  reform  activities. 

Exhibit  4.8a  shows  that,  according  to  our  national  sample  of  district  Eisenhower 
coordinators,  only  a minority  of  teachers  are  in  districts  that  offer  each  of  the  reform  types  of 
professional  development,  and  districts  vary  substantially  in  which  types  of  reform  professional 
development  they  support  with  Eisenhower  funds.  During  our  telephone  interviews,  we  asked 
Eisenhower  coordinators  whether  their  district  supported  participation  in  each  type  of  reform  activity, 
from  July  1 through  December  1997,  and  then  asked  whether  each  activity,  at  least  in  part,  was 
supported  with  Eisenhower  funds.  As  Exhibit  4.8a  illustrates,  over  one-third  of  teachers  (37  percent) 
are  in  districts  that  use  Eisenhower  funds  to  support  teacher  networks,  mentoring  or  coaching,  and 
committees  or  task  forces.  A smaller  percent  of  teachers  are  in  districts  that  support  study  groups  (21 
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percent),  internships  (four  percent)  and  individual  research  projects  (seven  percent).  Thirty-five 
percent  of  teachers  are  in  districts  that  do  not  support  any  type  of  reform  activity  (results  not  shown). 

EXHIBIT  4.8a 


Percent  of  Teachers  in  Districts  That  Support  Reform  Types  of 
Professional  Development  Activities  with  Eisenhower  Funds  (n=353) 


W 

k. 

0 

.C 

O 

cc 

0 


c 

0 

o 

v. 

0 

CL 


group  network  task  force  research 

project 


Types  of  reform  activities 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  21  percent  of  teachers  are  in  districts  that  use  Eisenhower  funds  to  support  teacher 
study  groups.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 


Although  nearly  two-thirds  of  teachers  are  in  districts  that  are  trying  reform  activities  (data 
not  shown),  relatively  few  teachers  participate  in  them.  We  asked  coordinators  to  tell  us  the  number 
of  teachers  that  Eisenhower  funds  supported  to  participate  in  each  type  of  reform  activity,  from  July 
1 through  December  1997.  As  Exhibit  4.8b  shows,  according  to  district  Eisenhower  coordinators 
nationwide,  none  of  the  six  reform  types  of  professional  development  activities  accounts  for  an 
average  of  more  than  five  percent  of  all  participations  in  all  Eisenhower-assisted  activities.  Although 
an  average  of  four  percent  and  seven  percent  of  teachers  are  in  districts  that  offer  internships  and 
individual  research  projects,  respectively  (as  illustrated  in  Exhibit  4.8a),  Exhibit  4.8b  shows  that 
average  teacher  participations  in  these  activities  is  less  than  one-half  of  one  percent.  As  we 
suggested  earlier,  however,  the  number  of  participations  in  reform  approaches  may  not  reflect  the 
proportion  of  funds  devoted  to  these  reform  types  of  activities. 

Exhibit  4.8b  also  illustrates  that  some  districts  rely  much  more  heavily  than  others  on  these 
reform  types  of  activities.  Some  districts  report  that  the  majority  of  teachers  who  participate  in 
Eisenhower-funded  activities  participate  in  reform  types  of  professional  development.  For  example, 
some  districts  report  as  many  as  60  to  70  percent  of  Eisenhower-assisted  participations  on  teacher 
committees  or  in  study  groups.  However,  many  districts  have  participations  well  below  the  average, 
including  districts  that  have  no  participations  in  any  reform  activities. 


erIc 


4-24 


EXHIBIT  4.8b 

Percent  of  Participations  in  Reform  Types  of  Eisenhower-assisted  Professional 

Development  Activities  (n=353) 


project 


Types  of  reform  activities 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  five  percent  of  participations  in  Eisenhower- 
assisted  activities  are  in  study  groups.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the 
dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  districts  for  that  particular  category.  The 
number  to  the  right  of  the  distribution  is  the  mean. 


Adding  the  total  number  of  participations  in  all  six  of  the  reform  types  of  professional 
development  provides  a measure  of  the  proportion  of  Eisenhower-assisted  participations  in  reform 
versus  traditional  approaches  to  professional  development.  Exhibit  4.9a  shows  that  the  district 
average  for  teacher  participations  in  reform  types  of  professional  development  is  16  percent.  There  is 
substantial  variation  on  this  measure,  however;  a few  districts  have  close  to  half  of  total 
participations  in  reform  approaches,  while  many  others  have  no  participations  in  reform  approaches. 
Further,  the  interaction  effect  between  poverty  and  size  is  significant. 

The  main  message  illustrated  by  the  interactions  in  Exhibit  4.9b  seems  to  be  that  while  high- 
and  medium-poverty  districts  have  more  participations  in  reform  activities  as  the  size  of  the  district 
increases,  low-poverty  districts  have  the  same  relatively  low  percent  of  participations  in  reform 
activities  regardless  of  the  size  of  the  districts  Reform  participations  in  consortia  mirror  the  pattern  in 
large  districts,  except  in  high-poverty  districts,  where  consortia  have  fewer  participations  in  reform 
activities  than  do  large  districts. 
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EXHIBIT  4.9a 


Percent  of  Participations  in  Reform  Types  of  Eisenhower-assisted  Professional 
Development  Activities,  Overall  and  by  District  Poverty  and  District  Size  (n=353) 
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Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  16  percent  of  all  participations  are  in  reform 
types  of  professional  development.  The  interaction  effects  of  district  poverty  and  size  on  the  percent  of  participations  in  reform 
activities  are  significant.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the  dots 
form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  districts  for  that  particular  category. 

The  number  to  the  right  of  the  distribution  is  the  mean. 


EXHIBIT  4.9b 

Percent  of  Teacher  Participations  in  Reform  Types  of  Professional  Development 
Activities,  Interaction  of  District  Poverty  and  District  Size  (n=353) 


Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  data  point  designated  by  the  first  square  indicates  that  the  average  percent  of  participations  in  reform  activities 
in  medium-poverty  small  districts  is  1 9 percent.  The  line  with  data  points  designated  by  diamonds  indicates  the  percent  of  participations  in 
reform  activities  for  high-poverty  districts  in  each  of  four  sizes/types  of  districts  (i.e.,  consortia,  large,  medium,  and  small  districts);  the  line 
with  data  points  designed  by  squares  indicates  the  percent  of  participations  in  reform  activities  for  medium-poverty  districts  for  each  of  the 
four  sizes/types  of  districts;  and  the  line  with  data  points  designated  by  triangles  indicates  the  percent  of  participations  in  reform  activities  in 
low-poverty  districts  for  each  of  the  four  sizes/types  of  districts. 
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This  may  reflect  the  fact  that  large  districts  are  more  likely  to  have  teachers  available  who 
have  the  time  to  participate  in  reform  activities,  which  sometimes  span  a longer  period  of  time  than 
traditional  activities;  that  is,  large  districts  are  more  likely  to  have  department  leaders  and/or 
curriculum  specialists  or  other  positions  that  do  not  require  100  percent  classroom  time,  whereas 
districts  with  smaller  numbers  of  teachers  may  be  less  likely  to  have  these  positions.  The  findings 
also  may  reflect  the  fact  that  reform  activities  typically  cost  more  per  teacher  than  traditional 
activities.  Large  districts  may  have  more  resources  available  per  teacher,  in  addition  to  benefiting 
from  economies  of  scale,  both  of  which  might  enable  them  to  offer  more  reform  activities,  compared 
to  smaller  districts.  Low-poverty  districts  generally  do  not  provide  many  opportunities  for 
participation  in  reform  activities,  possibly  because,  on  average,  their  students  perform  better  than  in 
other  districts.  As  a result,  low-poverty  districts  may  not  feel  the  need  to  offer  reform  approaches  to 
professional  development. 

The  total  number  of  types  of  activities  that  a district  funds  is  another  measure  of  a district’s 
portfolio  of  professional  development  activities.  This  measure  reflects  a district’s  emphasis  on 
reform  types  of  professional  development  and  allows  us  to  compare  how  districts  vary  in  their 
support  of  these  types  of  professional  development.  Since  almost  all  districts  support  the  two 
traditional  types  of  professional  development  activities  (i.e.,  in-district  workshops  and  institutes,  and 
out-of-district  workshops  and  conferences),  the  larger  the  number  of  types  of  activities  that  the 
district  supports,  the  more  likely  it  is  that  they  are  supporting  reform  types  of  professional 
development  activities.  Exhibit  4.10  shows  the  percent  of  teachers  in  districts  reporting  that  they  use 
Eisenhower  funds  to  support  different  numbers  of  types  of  activities.  We  created  this  measure  by 
adding  the  number  of  types  of  activities  that  district  coordinators  said  that  they  supported,  at  least  in 
part,  with  Eisenhower  funds,  from  July  1 through  December  3 1,  1997.  Of  a possible  ten  types  of 
activities  (the  two  traditional  and  six  reform  that  we  address  in  this  chapter,  plus  courses  and  teacher 
resource  centers),  districts  support  an  average  of  3.4  types  of  activities  with  Eisenhower  funds. 

Three  percent  of  teachers  are  in  districts  that  support  only  one  type  of  activity  (data  not  shown). 

Consistent  with  our  previous  findings  on  across-district  variation,  some  districts  use 
Eisenhower  funds  to  support  very  few  types  of  activities  during  the  time  period  in  question,  while 
some  support  as  many  as  eight  of  the  different  types.  Further,  low-poverty  districts  have 
significantly  fewer  types  of  activities  than  medium-  or  high-poverty  districts.  In  addition,  as  one 
would  expect,  small  districts  support  fewer  types  of  activities  than  either  medium  or  large  districts  or 
consortia,  and  medium-sized  districts  support  fewer  activities  than  large  districts.  Perhaps  the  higher 
the  poverty,  the  greater  the  perceived  need  to  experiment  with  multiple  and  new  forms  of 
professional  development;  and  the  larger  the  district,  the  greater  capacity  to  do  so. 
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EXHIBIT  4.10 


Number  of  Types  of  Eisenhower-assisted  Activities,  Overall  and  by  District  Poverty 

and  District  Size  (n=353) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  they  offer  3.4  out  of  a possible  1 0 activities,  with 
no  districts  offering  more  than  eight  types  of  activities.  The  number  of  types  of  Eisenhower-assisted  activities  differs  significantly  by  both 
district  poverty  and  district  size.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the  dots 
form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  the  number  of  types  of  activities  supported  by 
Eisenhower  funds.  The  number  to  the  right  of  the  distribution  is  the  mean.  • 


The  Structural  Features  of  Reform  Types  of  Activities 

Even  when  districts  support  reform  types  of  professional  development  activities,  they  may 
not  have  other  features  such  as  long  duration  that  would  foster  better  teacher  outcomes.  As  we 
suggested  earlier  in  this  report,  recent  literature  has  described  certain  reform  types  of  professional 
development  as  having  features  that  enable  teachers  to  reflect  on  their  new  knowledge,  practice  new 
skills,  and  engage  in  in-depth  discussions  of  implementation  issues.  This  implies  that  these  reform 
types  of  professional  development  occur  over  a period  of  time  that  allows  for  such  reflection, 
practice,  and  discussion. 

Span.  We  did  not  ask  coordinators  the  number  of  hours  that  teachers  spent  in  reform 
activities,  since  for  most  of  the  activities  a measure  of  hours  did  not  seem  easy  for  coordinators  to 
know;  for  example,  the  literature  suggests  that  participation  in  teacher  networks  often  is  ongoing. 

We  did  ask  Eisenhower  coordinators  to  report  the  time  period  over  which  each  type  of  reform 
activity  was  typically  spread;  the  choices  were  1)  one  month  or  less,  2)  more  than  one  month  but  less 
than  six  months,  3)  between  six  months  and  one  year,  or  4)  more  than  one  year.  For  the  analysis,  we 
combined  the  two  middle  categories.  Exhibit  4.1  la  shows  the  percentage  of  districts  reporting 
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“typical”  Eisenhower-assisted  reform  activities  that  lasted  less  than  one  month,  one  month  to  a year, 
and  more  than  a year. 

While  reform  activities  typically  extend  over  a relatively  long  period,  this  is  not  always  the 
case.  Exhibit  4.1  la  shows  that  Eisenhower  coordinators  report  that  nearly  half  (44  percent)  of  all 
teachers  are  in  school  districts  where  “typical”  study  groups  last  less  than  one  month,  and  one  quarter 
(25  percent)  of  teachers  are  in  school  districts  where  typical  collaboratives  or  networks  last  less  than 
one  month.  Such  activities  are  generally  described  in  the  literature  as  ongoing,  often  regularly 
scheduled  opportunities  for  teachers  to  meet  to  discuss  students’  learning  and  teacher  practice 
(Lieberman  & McLaughlin,  1992;  Little,  1993).  Therefore,  it  seems  surprising  that  such  a relatively 
high  percentage  of  teachers  are  in  districts  that  report  that  typical  Eisenhower-assisted  activities  of 
these  types  last  for  less  than  a month.  However,  a majority  of  teachers  are  in  districts  that  support 
reform  activities  that  typically  last  for  more  than  a month,  with  the  exception  of  internships.  In  the 
case  of  teacher  networks,  over  a quarter  of  teachers  (29  percent)  are  in  districts  that  typically  offer 
this  activity  for  longer  than  a year. 

EXHIBIT  4.11a 

Percent  of  Teachers  in  Districts  by  Span  of  Eisenhower-assisted  Reform  Activities 

(n  varies  by  type) 
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group  network  or  task  research 

force  project 

Types  of  “reform"  activities 


Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  44  percent  of  teachers  are  in  districts  that  have  study  groups  that  typically  last  less  than  one 
month,  39  percent  of  teachers  are  in  districts  that  have  study  groups  that  typically  last  between  one  and  12  months,  and  16  percent  of  teachers 
are  in  districts  that  have  study  groups  that  typically  last  more  than  12  months.  Each  bar  shows  the  percent  of  teachers  in  districts  that  report 
having  a particular  activity  for  each  of  the  three  time  span  categories,  illustrated  in  the  key.  The  number  at  the  top  of  each  shaded  area  is  the 
percent  of  teachers  in  districts  for  the  corresponding  time  span  category. 


4-29 


We  created  a composite  variable  of  the  span  of  all  activities,  both  traditional  and  reform, 
where  l=less  than  one  month,  2=one  to  12  months,  and  3=more  than  one  year.  As  Exhibit  4.1  lb 
shows,  professional  development  activities  in  large  districts  extend  for  a significantly  longer  span  of 
time  than  activities  in  small  districts.  There  are  no  statistically  significant  differences  for  district 
poverty  level.  These  results  support  the  notion  that  the  greater  capacity  of  larger  districts  may  enable 
them  to  design  and  offer  activities  that  span  a greater  length  of  time. 

EXHIBIT  4.11b 

Average  Span  of  Eisenhower-assisted  Activities,  Overall  and  by  District  Poverty  and 

District  Size(n=353) 


Not 

Significant 


1+1.28 


;+  1.39 


+ 1.50 


♦ + 1.40 


Overaii 


Low  Medium  High 


Small  Medium  Large  Consortium 


District  Poverty 


District  Size 


Significant  Pairwise  Contrasts 

Size Small  vs.  Large 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  their  Eisenhower-assisted  activities  have  a span 
of  1 .4  (i.e.,  between  “less  than  one  month”  and  “one  month  to  a year”).  The  span  of  traditional  and  reform  activities  differs  significantly  by 
district  size,  but  not  by  district  poverty  level.  Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value) 
increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  of  the  number  of  types  of 
activities  supported  by  Eisenhower  funds.  The  number  to  the  right  of  the  distribution  is  the  mean. 


Case-study  Examples  of  District  Support  for  Reform  Types  of  Professional 
Development  Activities 

Survey  data  show  that  some  districts  use  their  Eisenhower  funds  predominantly  for  reform 
types  of  professional  development  activities,  and  our  case-study  data  provide  us  with  examples  of 
districts.  Boonetown,  Kentucky,  one  of  our  in-depth  case  studies,  is  an  example  of  a district  in  which 
the  great  majority  of  Eisenhower-assisted  professional  development  supports  reform  types  of 
professional  development. 

In  Boonetown,  Eisenhower  funds  support  teacher  mentoring  and  coaching,  as  well  as  other 
forms  of  activities  where  teachers  share  knowledge  with  their  peers.  Almost  all  Eisenhower  funds  in 
the  district  go  toward  building  school  capacity.  The  district  does  this  by  supporting  four  resource 
teachers  who  act  as  mentors  to  other  teachers  in  the  district.  These  resource  teachers  provide  in-class 
modeling,  assistance  in  preparing  lessons,  and  observations  of  other  teachers.  In  each  school,  the 
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principal  determines  priorities  for  the  resource  teachers,  based  on  the  school  goals  that,  in  turn, 
reflect  the  performance  of  students  on  the  statewide  assessment.  Even  where  funds  are  used  to 
enhance  the  skills  of  individual  teachers,  they  are  expected  to  teach  these  skills  to  other  teachers.  The 
principals  encourage  teachers  with  strong  leadership  qualities  and  specific  professional  development 
interests  to  use  Eisenhower  funds  to  develop  their  expertise  in  an  area  (e.g.,  graphing  calculators), 
then  return  to  the  school  and  train  their  peers.  Teachers  may  use  Eisenhower  funds  to  attend 
conferences  as  well  as  workshops  if  they  will  share  their  knowledge  with  other  teachers  in  their 
schools. 

Comparing  Eisenhower-assisted  Activities  to  All  District  Professional 
Development  Activities 

So  far  we  have  been  focusing  on  district  portfolios  of  Eisenhower-assisted  professional 
development  activities.  But  Eisenhower-assisted  professional  development  is  only  one  part  of  a 
district’s  professional  development  program.  Title  II  provides  only  a portion  of  a district’s  budget 
and  incentives  for  professional  development.  Each  district  has  its  larger  portfolio  of  professional 
development  that  includes  both  Eisenhower-assisted  activities  and  non-Eisenhower  funded  activities. 
Most  school  districts  offer  teachers  opportunities  for  professional  development  that  they  fund  from 
state,  local,  or  other  federal  sources.  Our  survey  data  allow  us  to  further  examine  the  whole  pattern  of 
district  professional  development  activities,  both  those  funded  by  Eisenhower  and  those  funded  by 
other  sources.10 

We  asked  district  coordinators  whether  their  districts  offer  each  type  of  professional 
development  activity,  and  if  so,  if  they  support  the  activity  with  funds  from  the  Eisenhower  program. 
As  Exhibit  4.12  illustrates,  over  80  percent  of  teachers  work  in  districts  that  support  in-  and  out-of- 
district  workshops  and  use  Eisenhower  funds  to  do  so.  Less  than  10  percent  of  teachers  are  in 
districts  that  support  these  workshops  but  do  not  use  Eisenhower  funds  for  them. 

As  illustrated  in  Exhibit  4.13,  many  districts  offer  reform  types  of  professional  development 
activities  but  do  not  use  the  Eisenhower  funds  to  support  them.  For  example,  of  those  teachers  in 
districts  that  support  mentoring  and  committee  or  task  force  membership,  about  half  are  in  districts 
that  use  Eisenhower  funds  to  do  so.  That  is,  30  percent  of  teachers  are  in  districts  that  use 
Eisenhower  resources  to  fund  mentoring,  and  31  percent  of  teachers  are  in  districts  that  support 
mentoring,  but  not  with  Eisenhower  funds.  Similarly,  35  percent  of  teachers  are  in  districts  that  fund 
committees  or  task  forces  with  Eisenhower  resources,  and  35  percent  of  teachers  are  in  districts  that 
offer  these  activities,  but  do  not  use  Eisenhower  funds  for  them. 


10  We  explored  a number  of  analyses  to  identify  clusters  of  districts  that  were  similar  in  their  portfolios  of 
professional  development,  but  for  the  most  part  we  did  not  find  any  clear  clusters;  instead  there  appeared  to  be  great 
variability  among  districts  in  the  kinds  of  professional  development  that  they  provide.  We  were  therefore  unable  to 
classify  districts  according  to  the  types  or  characteristics  of  the  professional  development  that  they  provide. 
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EXHIBIT  4.12 


Percent  of  Teachers  in  Districts  That  Fund  Workshops  with  Eisenhower  Funds,  Other 

Sources,  or  Not  at  All  (n=353) 
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Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  7 percent  of  teachers  are  in  districts  that  do  not  offer  in-district  workshops  and 
institutes,  3 percent  of  teachers  are  in  districts  that  offer  them  but  do  not  support  them  with  funds  from  the  Eisenhower  program,  and  89 
percent  of  teachers  are  in  districts  that  offer  them  and  support  them  with  Eisenhower  funds.  Each  bar  shows  the  percent  of  teachers  in 
districts  that  report  the  status  of  a particular  type  of  activity,  for  each  of  the  three  status  categories  illustrated  in  the  key.  The  number  at 
the  top  of  each  shaded  area  is  percent  of  teachers  in  districts  for  the  corresponding  status  category. 


Eisenhower  support  for  the  remaining  four  reform  activities  varies.  Districts  that  support 
teacher  networks  are  more  likely  than  not  to  use  Eisenhower  funds  to  do  so;  thirty-seven  percent  of 
teachers  are  in  districts  that  fund  teacher  networks  with  Eisenhower  dollars,  and  24  percent  are  in 
districts  that  support  networks,  but  do  not  use  Eisenhower  funds  for  them.  In  contrast,  districts  are 
less  likely  to  use  Eisenhower  funds  to  support  study  groups,  internships,  and  individual  research 
projects.  For  example,  21  percent  of  teachers  are  in  districts  that  use  Eisenhower  funds  to  support 
study  groups,  while  28  percent  of  teachers  are  in  districts  that  support  study  groups  without  using 
Eisenhower  funds.  Very  few  districts  use  money  from  the  Eisenhower  program  to  support 
internships  and  research  projects.  Only  four  percent  of  teachers  are  in  districts  that  use  Eisenhower 
money  to  support  internships,  compared  to  17  percent  of  teachers  in  districts  that  support  this  activity 
without  using  Eisenhower  funds.  The  same  pattern  exists  for  individual  research  projects;  only  seven 
percent  of  teachers  are  in  districts  that  support  research  projects  with  Eisenhower  funds,  while  19 
percent  of  teacher  are  in  districts  that  support  them  but  do  not  use  Eisenhower  funds  to  do  so. 

Although  Eisenhower  funds  may  be  used  to  provide  support  for  all  types  of  professional 
development,  districts  are  much  more  likely  to  report  using  Eisenhower  resources  to  fund  traditional 
types  of  professional  development — specifically  in-  and  out-of-district  workshops  and  institutes — 
than  any  other  type  of  activity.  Teacher  networking  is  the  only  type  of  reform  activity  for  which 
more  than  half  of  teachers  in  districts  that  offer  the  activity  are  in  districts  that  use  Eisenhower  funds 
to  support  it. 
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EXHIBIT  4.13 

Percent  of  Teachers  in  Districts  That  Fund  Reform  Types  of  Activities  with 
Eisenhower  Funds,  Other  Sources,  or  Not  at  All  (n=353) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  52  percent  of  teachers  are  in  districts  that  do  not  offer  study  groups;  28  percent  of 
teachers  are  in  districts  that  offer  them  but  do  not  support  them  with  Eisenhower  funds;  and  21  percent  of  teachers  are  in  districts  that  offer 
them  and  support  them  with  Eisenhower  funds.  Each  bar  shows  the  percent  of  teachers  in  districts  that  report  the  status  of  a particular 
activity,  for  each  of  the  three  categories  illustrated  in  the  key.  The  number  at  the  top  of  each  shaded  area  is  percent  of  teachers  in  districts 
for  the  corresponding  status  category. 


Given  the  goals  of  the  Eisenhower  program  to  support  professional  development  that  is 
sustained,  intensive,  and  of  high  quality,  and  given  our  finding  in  Chapter  3 that  reform  types  of 
professional  development  are  associated  with  other  structural  and  core  features  of  high-quality,  these 
findings  may  be  cause  for  concern.  They  may  indicate  that  many  districts  do  not  view  Eisenhower 
funds  as  a resource  for  precisely  the  types  of  activities  that  the  legislation  is  intended  to  support." 

It  is  unclear  why  districts  may  choose  to  use  Eisenhower  funds  disproportionately  for 
workshops  rather  than  for  other  types  of  activities.  Districts  may  choose  to  spend  Eisenhower  money 
on  workshops  for  the  same  reason  that  workshops  are  popular  generally — because  this  method  allows 
districts  to  reach  more  teachers  for  less  money  than  would  be  possible  with  most  of  the  reform  types 
of  activities.  Another  possibility  is  that  several  of  the  reform  activities,  such  as  mentoring  and 
intemship/immersion  activities,  require  resources  that  are  greater  than  those  available  through  the 
Eisenhower  program.  In  such  cases,  districts  may  use  other  sources  of  funding  for  these  extended, 
more  expensive  activities,  and  use  Eisenhower  funds  for  add-on  programs  and  activities,  such  as 


Our  data  do  not  provide  information  on  the  proportion  of  within  district  funds  spent  on  traditional  vs.  reform 
activities. 
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workshops  and  institutes,  which  are  less  typically  less  expensive.  Further,  systems  may  already  be  in 
place  that  channel  Eisenhower  funds  to  these  traditional  professional  development  activities  before 
any  other  type  of  activities,  and  these  systems  may  be  difficult  to  alter. 

Finally,  districts  may  be  unaware  that  they  can  use  Eisenhower  funds  for  some  reform  types 
of  professional  development.  In  response  to  questions  about  whether  they  used  Eisenhower  funds  for 
reform  types  of  professional  development  activities,  some  district  coordinators  in  our  national  sample 
appeared  surprised  that  they  could  use  Eisenhower  funds  for  these  purposes. 

Summary:  Structural  and  Core  Features  of  Traditional  and  Reform 
Activities 

Findings  from  our  National  Profile  of  Eisenhower  coordinators  and  from  our  case-study 
districts  clearly  indicate  that  districts  differ  substantially  in  the  patterns  of  activities  that  they  support 
with  Eisenhower  funds.  Data  from  district  Eisenhower  coordinators  indicate  that  most  school  districts 
use  Eisenhower  funds  to  support  traditional  types  of  professional  development — that  is,  in-district 
workshops  and  institutes,  and  out-of-district  workshops  and  conferences.  According  to  district 
Eisenhower  coordinators,  these  activities  generally  are  eight  hours  or  less,  and  span  over  less  than  a 
week.  In  most  districts,  the  great  majority  of  participations  are  in  such  traditional  types  of 
professional  development  activities.  Further,  district  coordinators  also  report  that  the  opportunity  for 
active  learning  that  they  are  least  likely  to  offer  is  teachers  being  observed  in  their  own  classrooms, 
which  may  be  one  of  the  most  important  methods  for  changing  teacher  practice. 

However,  data  from  district  coordinators  indicate  that  districts  vary  tremendously  in  their 
reliance  on  traditional  types  of  professional  development  activities  and  in  the  quality  of  these 
activities.  While  most  districts  have  large  proportions  of  participations  in  such  activities,  other 
districts  do  not.  Some  districts  support  workshops  that  have  high-quality  structural  and  core 
features — by  involving  greater  amounts  of  learning  time,  more  collective  participation,  and 
opportunities  for  active  learning,  for  example — while  workshops  in  other  districts  are  less  likely  to 
have  these  features. 

Similarly,  while  many  districts  use  Eisenhower  funds  to  support  teachers’  participation  in 
reform  types  of  professional  development  activities,  teacher  participation  in  these  reform  types  is 
typically  very  low.  This  is  even  more  true  in  small  and  low-poverty  districts. 

This  pattern  may  be  partially  explained  by  the  fact  that  several  of  these  reform  activities  do 
not  lend  themselves  to  large  numbers  of  participants,  but  rather  are  designed  specifically  for  small 
numbers  of  participants,  such  as  teacher  committees.  Districts  also  may  not  be  able  to  afford  support 
for  some  of  the  reform  types  of  professional  development  for  large  numbers  of  their  teachers.  By 
their  nature,  some  reform  types  of  professional  development  may  be  more  costly  than  traditional 
types  of  professional  development,  or  may  take  more  time  than  teachers  have  to  commit.  For 
example,  if  districts  were  to  pay  teachers  for  the  time  involved  in  formal  mentoring  relationships  that 
extend  over  long  periods  of  time,  or  for  ongoing  study  groups,  or  for  substitutes  so  that  teachers 
could  participate  in  such  activities,  they  would  certainly  be  more  costly  than  traditional  workshop 
arrangements. 

District  coordinators’  reports  about  patterns  of  support  for  reform  activities  mirror  the  data 
from  our  teacher  survey,  reported  in  Chapter  3.  While  a notable  number  of  school  districts  use 
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Eisenhower  funds  for  reform  types  of  professional  development  activities,  most  districts  tend  to 
support  such  activities  for  only  a small  proportion  of  their  teachers. 

These  findings  appear  to  indicate  that  there  is  room  for  districts  to  improve  in  meeting  the 
intent  of  the  1994  reauthorization.  However,  in  the  absence  of  information  about  the  use  of 
Eisenhower  funds  for  reform  types  of  professional  development  activities  prior  to  the  1994 
reauthorization,  we  do  not  know  for  sure  whether  districts  are  moving  in  the  direction  of  using 
Eisenhower  funds  for  reform  activities.  The  previous  evaluation  of  the  Eisenhower  program  did  not 
ask  a national  sample  of  districts  about  their  support  of  reform  types  of  activities  (Knapp  et  al.  1991). 
If  such  activities  were  supported  very  rarely  in  the  past,  then  the  proportions  that  we  report  could 
represent  an  increase  in  districts’  reliance  on  reform  types  of  professional  development  activities 
supported  by  Eisenhower  funds. 

Further,  we  found  that  not  all  of  the  traditional  activities  have  characteristics  of  low-quality 
professional  development,  nor  are  all  reform  types  of  high  quality.  The  literature  on  professional 
development  suggests  that  traditional  types  of  professional  development  generally  provide  less 
opportunity  for  reflection,  practice,  and  discussion  than  do  reform  types  of  professional  development 
(Little,  1993;  Sparks  & Loucks-Horsley,  1989).  This  means  that,  in  general,  traditional  types  of 
activities  would  be  less  likely  than  reform  activities  to  foster  the  types  of  reflection,  in-depth 
treatment  of  content,  and  opportunities  for  active  learning  that  were  envisioned  in  the  1994 
reauthorization.  While  this  may  generally  be  true,  as  we  demonstrated  in  Chapter  3,  our  data  from 
district  Eisenhower  coordinators  indicate  that  the  duration  of  both  types  of  activities  varies 
substantially  across  districts.  In  some  districts,  traditional  types  of  professional  development  meet  for 
many  hours,  extend  over  a long  period  of  time,  include  multiple  learning  strategies,  and  have 
opportunities  for  collective  participation.  And  in  some  districts,  reform  types  of  professional 
development,  such  as  networks  or  study  groups,  last  only  a few  weeks.  The  range  across  districts  in 
the  duration  of  reform  types  of  activities,  and  the  fact  that  a substantial  proportion  of  these  activities 
are  spread  over  less  than  one  month,  are  surprising  findings.  They  lead  us  to  conclude  that  both 
traditional  and  reform  types  of  professional  development  can  have  characteristics  of  high-quality 
professional  development,  and  that  reform  types  of  professional  development  are  not  automatically 
of  higher  quality  than  professional  development  activities  with  a traditional  format. 

t 

Our  results  indicate  that  although  some  districts  provide  “sustained  and  intensive” 
professional  development,  whether  structured  as  a traditional  or  reform  activity,  most  districts  do  not. 
Thus  it  seems  that  the  legislative  intent  of  the  1994  reauthorization  has  not  yet  been  realized,  except 
in  a minority  of  sites.  Districts  do  not  adequately  use  their  Eisenhower  funds  to  support  professional 
development  activities  that  have  the  features  of  high  quality  as  identified  in  the  literature  on 
professional  development  and  in  the  findings  from  our  study  of  teachers  in  Chapter  3.  Although 
districts  provide  activities  of  longer  duration  than  in  the  early  1990s  (see  Knapp  et  al.,  1991),  our 
findings  suggest  that  there  is  still  a great  deal  of  room  for  improvement.  This  leads  us  to  the 
conclusion  that  federal  efforts  must  continue  to  encourage  districts  to  support  sustained,  intensive, 
and  high-quality  professional  development  activities  with  Eisenhower  funds. 
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Targeting  and  Recruitment  of  Teachers 


Section  Findings 

♦ Districts  report  that  they  target  professional  development  activities  more  to  teachers  in 
low-  achievement,  high-poverty,  and  Title  I schools  than  to  other  groups  of  teachers. 
High-poverty  districts  and  large  districts  target  these  groups  of  teachers  significantly 
more  than  other  districts. 

♦ Almost  all  teachers  come  to  participate  in  Eisenhower-assisted  activities  through 
volunteering  or  principal  selection. 

♦ Nearly  all  districts  publicize  their  professional  development  activities,  and  most  also  try 
to  increase  participation  by  tailoring  the  focus  of  activities  to  specific  groups  of  teachers 
and  creating  incentives  for  participation. 

So  far,  we  have  discussed  differences  in  professional  development  opportunities  among 
districts  according  to  their  emphasis  on  particular  subject  matter,  their  focus  on  traditional  versus 
reform  types  of  activities,  and  the  structure  and  core  experiences  of  these  activities.  Two  other 
important  factors  that  affect  teachers’  professional  development  opportunities  are  the  extent  to  which 
districts  target  activities  toward  specific  groups  of  teachers  and  their  methods  of  recruiting  teachers 
to  participate  in  activities.  In  Chapter  3,  we  found  that,  despite  the  program’s  formula  that  targets 
more  funding  to  higher  poverty  districts,  and  the  legislative  emphasis  on  serving  teachers  of  special 
populations  of  children,  such  teachers  were  only  somewhat  more  likely  to  participate  in  Eisenhower- 
assisted  professional  development  than  would  be  expected  from  their  numbers  in  the  population  as  a 
whole. 


The  Eisenhower  legislation,  like  other  federal,  state,  and  local  efforts,  aims  to  improve  the 
educational  outcomes  of  all  students,  including  those  who  historically  have  not  had  access  to  high- 
quality  educational  opportunities,  or  who  have  had  greater  difficulty  succeeding  in  school.  Several 
Title  II  provisions  emphasize  that  activities  for  teachers  funded  by  the  Eisenhower  program  should 
benefit  students  from  diverse  backgrounds.  The  law  states  as  one  of  its  purposes  the  goal  of  “meeting 
the  educational  needs  of  diverse  student  populations,”  including  economically  disadvantaged 
students  (Section  2002(2)(D)). 

In  addition,  the  local  plan  for  professional  development,  required  in  the  legislation,  must 
describe  how  local  professional  development  activities  meet  the  needs  of  these  students.  The  law 
contains  a number  of  provisions  that  share  a particular  emphasis  on  activities  that  meet  the  needs  of 
teachers  who  work  in  school  with  high  rates  of  poverty  and  low  achievement,  and  with  diverse 
student  populations. 

In  particular,  local  applications  for  Title  II  funds  must  include  a description  of  how  Title  II 
activities  will  be  designed  to  address  the  needs  of  teachers  who  are  working  in  schools  that  receive 
assistance  from  the  federal  government  under  Part  A of  Title  I (Section  2208(d)(1)(B)). 

The  rationale  for  these  provisions  is  that  teachers  of  special  populations  of  students  are  more 
likely  than  other  teachers  to  have  little  teaching  experience;  be  working  at  schools  with  fewer 
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resources;  have  larger  classes;  have  behavior,  safety,  and  other  non-academic  issues  to  confront;  and 
have  students  who  are  more  challenging  to  teach  than  more  advantaged  students  (Darling-Hammond, 
1997a;  U.S.  Department  of  Education,  1999a).  Teachers  of  disadvantaged  students  often  do  not  feel 
prepared  to  meet  the  needs  of  their  students  (U.S.  Department  of  Education,  1999a).  Thus,  these  are 
often  the  teachers  who  most  need  professional  development  (Corcoran,  1995;  Darling-Hammond, 
1997b).  Although  they  may  be  more  in  need  of  professional  development  than  others,  teachers  of 
students  in  schools  with  high  rates  of  poverty,  and  low-achievement,  or  Title  I schools,  sometimes 
have  less  opportunity  to  participate  in  certain  types  of  professional  development  than  teachers  of 
students  in  more  advantaged  schools  (U.S.  Department  of  Education,  1998a). 

In  this  section  of  the  chapter,  we  present  data  that  demonstrate  the  extent  to  which  districts 
address  the  legislative  emphasis  on  targeting  and  recruiting  specific  groups  of  teachers  to  participate 
in  Eisenhower-assisted  professional  development  opportunities.  First  we  show  how  districts  choose 
different  groups  of  teachers  to  target,  then  we  present  data  about  the  different  ways  that  teachers 
come  to  participate  in  Eisenhower  professional  development  activities.  Finally,  we  show  how 
districts  vary  in  the  type  and  number  of  methods  that  they  use  to  increase  teacher  participation  in 
Eisenhower-assisted  professional  development  opportunities. 

Targeting  Teachers  of  Special  Populations  of  Students 

The  district  can  influence  who  participates  in  professional  development  activities  in  several 
ways.  One  way  is  by  focusing  professional  development  opportunities  on  a particular  subject  area. 

As  we  illustrated  in  the  first  section  of  this  chapter,  virtually  all  school  districts  target  their 
Eisenhower-assisted  activities  on  mathematics  and  science  and,  therefore,  target  teachers  of  these 
subjects. 

Another  way  districts  influence  what  type  of  teachers  participate  is  by  focusing  district 
recruitment  efforts  on  specific  groups  of  teachers.  To  find  out  whether  districts  are  targeting  teachers 
of  special  needs  and  disadvantaged  students  as  outlined  in  the  legislation,  we  asked  district 
Eisenhower  coordinators  to  indicate  whether  they  placed  “no  particular  emphasis,”  “some  emphasis,” 
or  a “strong  emphasis”  on  recruiting  the  following  types  of  teachers:  1)  teachers  from  Title  I schools, 
2)  special  education  teachers,  3)  teachers  of  limited  English  proficiency  students,  4)  teachers  from 
schools  with  low  achievement  levels,  and  5)  teachers  from  high-poverty  schools  (50  percent  or  more 
students  eligible  for  free/reduced-price  lunch).12 

As  Exhibit  4. 14  illustrates,  similar  percentages  of  teachers  are  in  districts  that  report  placing 
some  or  a strong  emphasis  on  recruiting  teachers  from  Title  I schools  (58  percent ),  schools  with  low 
achievement  (66  percent),  and  high-poverty  schools  (58  percent).  Sixty  percent  of  teachers  are  in 
districts  where  the  Eisenhower  coordinator  reports  placing  some  or  a strong  emphasis  on  recruiting 
special  education  teachers,  and  47  percent  of  teachers  are  in  districts  that  place  an  emphasis  on 
recruiting  teachers  of  students  with  limited  English  proficiency.  Thus,  a majority  of  district 
coordinators  say  that  they  are  targeting  their  professional  development  programs  to  the  particular 
groups  of  teachers  emphasized  in  the  legislation. 


12  A teacher  can  work  at  a high-poverty  school  whether  or  not  he  or  she  is  in  a high-poverty  district.  A district  is 
categorized  as  high-poverty  based  on  the  percent  of  low-income  students  in  the  district;  within  a district,  schools 
vary  in  the  number  of  low-income  students  who  attend  them,  and  low- income -students  are,  often  concentrated  in 
particular  schools  within  a district. 
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EXHIBIT  4.14 


Percent  of  Teachers  in  Districts  That  Report  Placing  No,  Some,  or  a Strong  Emphasis 
on  Recruiting  Teachers  of  Special  Student  Populations  (n=363) 
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education  students  achievement  schools 

students  schools 

Teachers  by  type  of  school  or  type  of  students 


□ None 

□ Some 
■ Strong 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  42  percent  of  teachers  are  in  districts  that  place  no  emphasis  on  recruiting  teachers  in 
Title  I schools,  28  percent  of  teachers  are  in  districts  that  place  some  emphasis  on  recruiting  these  teachers,  and  30  percent  of  teachers  are 
in  districts  that  place  a strong  emphasis  on  recruiting  them.  Each  bar  shows  the  percent  of  teachers  in  districts  that  report  placing  “none,” 
“some,”  or  a “strong”  emphasis  (as  illustrated  in  the  key)  on  recruiting  teachers  of  special  populations  of  students.  The  number  at  the  top 
of  each  shaded  area  is  percent  of  teachers  in  districts  for  the  corresponding  category  of  emphasis. 


Further  analysis  of  these  data  indicates  that  districts  that  emphasize  recruiting  teachers  of  one 
special  population  group  (e.g.,  special  education  teachers)  tend  to  emphasize  recruiting  teachers  of 
other  special  populations  as  well.  As  Exhibit  4.15  illustrates,  correlations  of  the  emphasis  given  to 
recruiting  different  groups  of  teachers  range  from  a moderate  .53  to  a high  of  .8.  (A  coefficient  of 
one  indicates  perfect  correlation.)  In  other  words,  some  districts  tend  to  emphasize  recruiting 
multiple  types  of  teachers;  other  districts  do  not,tend  to  target  at  all. 

We  formed  a scale  to  measure  the  extent  of  district  targeting  efforts  by  adding  district 
responses  to  how  much  emphasis  they  placed  on  recruiting  the  different  groups  of  teachers,  where 
l=“no  particular  emphasis,”  2=“some  emphasis,”  and  3=“strong  emphasis.”  As  Exhibit  4.16  shows, 
the  extent  to  which  districts  try  to  recruit  various  groups  of  teachers  varies  significantly  by  both 
district  poverty  and  size.  High-poverty  districts  are  more  likely  to  recruit  teachers  of  special 
populations  than  either  low-  or  medium-poverty  districts.  Similarly,  large  districts  are  more  likely  to 
recruit  particular  types  of  teachers  than  small-  or  medium-sized  districts. 
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EXHIBIT  4.15 


Correlations  Among  the  Emphasis  on  Recruitment  of  Teachers  of  Special 

Populations  of  Students  (n=363) 


Special  Education 
Teachers 

Teachers  of  Limited 
English  Students 

Teachers  from  Low- 
Achievement  Schools 

Teachers  from  High- 
Poverty  Schools 


Teachers 
from  Title  i 

Special 

Education 

Teachers 

Teachers 
of  Limited 
English 
Students 

Teachers 
from  Low- 
Achievement 
Schools 

.65 

.56 

.62 

.63 

.60 

.53 

.70 

.56 

.60 

.80 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  correlation  shows  that  recruiting  special  education  teachers  and  recruiting  teachers  from 
Title  I schools  are  significantly  correlated  at  .65  where  one  indicates  a perfect  correlation.  Each  coefficient  indicates  the 
magnitude  of  the  correlation  between  the  two  variables  it  represents.  All  correlations  are  significant  at  the  .05  level. 

Note : The  coefficients  are  Pearson  correlation  coefficients. 


Districts’  reports  of  an  emphasis  on  recruiting  teachers  of  diverse  student  populations, 
especially  teachers  from  low-achievement  or  high-poverty  schools,  is  curious  in  light  of  the  findings 
of  the  previous  chapter.  In  Chapter  3,  we  found  that  teachers  from  high-poverty  schools,  or  schools 
with  high  proportions  of  minority  students,  are  only  somewhat  more  likely  than  other  teachers  to 
participate  in  Eisenhower-assisted  professional  development  activities.  This  is  the  case  despite  the 
fact  that  the  Eisenhower  formula  provides  a greater  amount  of  funding  to  districts  with  large 
proportions  of  poor  children.  Thus,  while  most  teachers  are  in  districts  that  report  placing  an 
emphasis  on  recruiting  teachers  of  students  from  diverse  populations,  teachers  in  high-poverty 
schools  are  not  much  more  likely  than  others  to  participate,  according  to  our  national  survey  of 
teachers.  To  explore  this  issue  further,  we  now  turn  to  a description  by  district  coordinators  of  how 
teachers  come  to  participate  in  Eisenhower-assisted  activities. 


EXHIBIT  4.16 


Extent  of  Recruitment  of  Teachers  of  Special  Populations  of  Students,  Overall  and  by 

District  Poverty  and  District  Size  (n=363) 
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Size Small  vs.  Large,  Medium  vs.  Large 


Source.  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  they  place  “some  emphasis”  (i.e.,  1.9,  where 
one=no  emphasis,  two=some  emphasis,  and  three=strong  emphasis)  on  recruiting  teachers  of  various  types  of  students.  The  extent  of 
targeting  differs  significantly  both  by  district  poverty  and  district  size.  Each  dot  represents  one  district.  As  the  number  of  participations 
at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  extent  of 
emphasis  that  districts  place  on  recruiting  various  types  of  teachers.  The  number  to  the  right  of  the  distribution  is  the  mean. 


How  Teachers  Come  to  Participate  in  Eisenhower-assisted  Professional 
Development  Activities 

Teachers  can  come  to  participate  in  Eisenhower-assisted  professional  development  activities 
in  a number  of  ways.  For  instance,  they  can  volunteer  to  participate,  they  can  be  selected  to  attend 
by  their  principals,  they  can  take  turns  participating,  or  they  can  be  selected  to  participate  by  the 
professional  development  provider.  An  evaluation  of  NSF’s  Statewide  Systemic  Initiatives  noted 
that  heavy  reliance  on  teachers  to  volunteer  for  high-quality  professional  development  activities 
is  “at  best  only  a part  of  a strategy  for  systemic  reform  of  mathematics  and  science  education” 
(Corcoran,  Shields,  & Zucker,  1998).  One  reason  is  that  districts  may  not  be  able  to  shape  the 
incentives  and  constraints  that  determine  which  teachers  volunteer.  Therefore  districts  that  rely  on 
other  methods  to  increase  teacher  participation  besides  volunteering  may  be  more  likely  to  access 
teachers  who  otherwise  would  not  participate.  With  this  in  mind,  we  asked  district  coordinators  to 
indicate  what  percent  of  the  teachers  in  Eisenhower-assisted  activities  come  to  participate  in  each  of 
the  following  ways:  1)  volunteering,  2)  selection  by  their  principal  or  other  administrator,  3) 
selection  by  providers,  4)  rotation,  and  5)  other  ways. 

Exhibit  4.17  illustrates  that  the  vast  majority  of  teachers  participate  in  Eisenhower-assisted 
professional  development  either  by  volunteering  (68  percent)  or  by  being  selected  by  the  principal 
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(24  percent).  The  exhibit  also  shows  that  districts  vary  tremendously  in  their  use  of  these  different 
methods  of  drawing  teachers  to  participate  in  Eisenhower-assisted  professional  development 
opportunities.  A good  number  of  districts  rely  exclusively  or  nearly  exclusively  on  volunteers.  And 
though  by  and  large  districts  do  not  rely  heavily  on  rotation  or  selection  by  provider  to  attract 
teachers  to  participate,  some  districts  use  these  methods  extensively. 

The  heavy  reliance  by  districts  on  volunteers  may  be  one  explanation  of  the  fact  that  teachers 
of  students  with  the  greatest  needs  do  not  participate  in  Eisenhower-assisted  activities  in  greater 
numbers,  despite  districts’  reported  emphasis  on  recruiting  such  teachers.  Analyses  (not  shown) 
indicate  that  teachers  in  high-poverty  schools  are  no  more  or  less  likely  to  volunteer  for  professional 
development  than  other  teachers;  thus,  it  may  be  more  effective  to  use  other  strategies  to  increase  the 
participation  of  teachers  of  high-need  students.  Designing  activities  for  whole  schools  may  be  one 
way  of  addressing  this  problem,  while  also  achieving  the  benefits  of  collective  participation. 

However,  a whole-school  approach  may  not  be  effective  in  some  cases;  successful  participation  in 
professional  development  activities  demands  a certain  level  of  commitment  by  teachers,  and 
recruiting  volunteers  helps  to  ensure  that  teachers  are  willing  to  devote  the  time  and  effort  required  to 
benefit  from  the  activity. 


EXHIBIT  4.17 

Percent  of  Participations  in  District  Eisenhower-assisted  Activities,  By  How  Teachers 

Come  to  Participate  (n=35813) 


Type  of  participation 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  districts  report  that  68  percent  of  teacher  participations  are  volunteers. 
Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in 
length.  Each  distribution  represents  the  distribution  of  districts  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the 
mean. 


13  Of  our  total  sample  of  363  district  Eisenhower  coordinators,  five  did  not  answer  this  survey  question;  therefore  the 
sample  size  for  this  variable  was  358. 
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Methods  of  Increasing  Teacher  Participation 

As  part  of  targeting  particular  groups  of  teachers  for  participation  in  Eisenhower-assisted 
professional  development  activities,  districts  can  take  a number  of  steps  to  ensure  that  teachers  have 
information  about  these  activities.  We  asked  district  coordinators  to  indicate  the  ways  in  which  they 
try  to  increase  the  participation  of  teachers,  paraprofessionals,  or  other  staff.  District  coordinators 
indicated  whether  they  1)  publicize  activities,  2)  tailor  the  focus  of  professional  development  toward 
the  needs  of  special  populations,  3)  use  incentives,  4)  use  other  strategies,  or  5)  use  no  special 
strategies.  The  question  did  not  ask  how  often  districts  used  these  strategies,  but  only  whether  they 
used  a particular  strategy. 

Exhibit  4.18  shows  the  percent  of  district  Eisenhower  coordinators  who  report  undertaking 
different  methods  to  increase  participation  in  Eisenhower-assisted  professional  development 
activities. 


EXHIBIT  4.18 

Percent  of  Teachers  in  Districts  That  Use  Various  Strategies  to  Increase  the 
Participation  of  Teachers  in  Eisenhower-assisted  Activities  (n=363) 


Publicize  Tailor  Use  incentives  Other  None  used 

activities  focus 


Types  of  strategies 


Source'.  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  96  percent  of  teachers  are  in  districts  that  try  to  increase  teacher  participation  by 
publicizing  their  professional  development  activities.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in 
districts  for  each  category. 


Most  districts  use  several  methods  to  increase  teacher  participation,  but  some  methods  are 
used  more  than  others.  Most  teachers  (96  percent)  are  in  districts  that  publicize  their  professional 
development  activities,  80  percent  of  teachers  are  in  districts  that  tailor  the  focus  of  the  activities  to 
special  populations,  and  72  percent  are  in  districts  that  use  incentives  to  increase  participation  in 
professional  development  activities.  The  finding  that  80  percent  of  teachers  work  in  districts  that 
report  tailoring  the  focus  of  professional  development  activities  toward  the  needs  of  special 


populations  goes  hand-in-hand  with  the  findings  reported  earlier  that  a majority  of  teachers  are  in 
districts  that  report  placing  either  some  or  a strong  emphasis  on  including  teachers  in  low- 
achievement,  high-poverty,  and  Title  I schools,  and,  to  a lesser  extent,  special  education  teachers  and 
teachers  of  LEP  students.  Nevertheless,  neither  developing  a tailored  focus,  nor  the  other  strategies 
designed  to  increase  the  participation  of  teachers,  appear  to  be  effective  in  fostering  the  actual 
participation  of  teachers  in  high-poverty  or  high-minority  schools,  as  reported  in  Chapter  3. 

Summary:  Targeting  and  Recruitment  of  Teachers 

Our  data  show  that  most  teachers  are  in  districts  that  report  making  an  effort  to  target 
teachers  of  students  in  schools  with  particular  risk  factors,  such  as  high  poverty  or  low  achievement. 
Further,  most  teachers  work  in  districts  that  use  several  strategies  for  informing  teachers  about 
professional  development  opportunities.  Despite  these  efforts,  and  despite  greater  funding  to  districts 
that  have  large  numbers  of  high-poverty  students,  teachers  of  students  in  high  poverty  are  only 
slightly  more  likely  than  others  to  participate  in  Eisenhower-assisted  activities.  These  findings 
question  the  actual  implementation  and  effectiveness  of  recruitment  strategies.  Although  in  our 
national  survey,  district  coordinators  reported  extensive  targeting,  in  our  six  exploratory  case  sites 
(see  Birman,  Reeve,  & Sattler,  1998),  Eisenhower  coordinators  followed  the  approach  of  making 
Eisenhower-assisted  activities  available  to  all  teachers,  without  making  special  efforts  to  target 
teachers  of  at-risk  children.  Our  in-depth  cases  also  did  not  find  explicit  strategies  to  recruiting 
teachers  from  high-poverty  schools,  or  teachers  of  other  at-risk  students.  These  apparently  mixed 
findings  suggest  that  more  in-depth  questioning  of  Eisenhower  coordinators  may  reveal  less 
implementation  of  targeting  and  recruiting  strategies  than  might  be  represented  by  our  telephone 
survey  results. 

Even  if  districts  do  engage  in  targeting  practices,  the  practices  do  not  seem  to  be  very 
effective.  One  reason  that  teachers  of  high-need  students  appear  not  to  participate  as  extensively  as 
district  targeting  strategies  might  suggest,  could  be  the  heavy  emphasis  that  districts  place  on 
teachers’  volunteering  for  professional  development  activities,  since  districts  may  be  restricted  in 
their  ability  to  influence  which  teachers  volunteer.  These  teachers  in  high-poverty  schools  may  feel 
less  comfortable  taking  time  away  from  their  students  to  attend  professional  development  activities, 
given  the  academic  challenges  that  their  students  face,  as  well  as  the  behavior,  safety,  and  non- 
academic  problems  that  they  may  confront. 

It  would  be  helpful  to  have  supplemental  data  that  provide  a measure  of  the  quality  of 
districts’  targeting  and  recruiting  efforts,  as  well  as  data  about  what  might  encourage  teachers  to 
volunteer  for  professional  development.  Even  without  this  information,  our  findings  suggest  that 
policymakers  and  program  administrators  should  increase  efforts  to  target  teachers  of  at-risk 
students,  and  develop  effective  methods  of  accomplishing  this  important  goal. 
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Differences  in  District  Portfolios  of  Eisenhower- Assisted 
Activities  by  District  Poverty  and  Size 


Throughout  this  chapter,  for  key  variables  we  have  indicated  where  patterns  of  Eisenhower 
support  for  professional  development  differ  significantly  according  to  the  district  poverty  level  or  the 
size  of  the  district.  As  we  noted  previously,  all  of  the  analyses  simultaneously  control  for  size  and 
poverty,  so  any  significant  effects  are  independent  of  one  another.  Taken  together,  these  findings 
help  us  understand  how  district  demographic  factors  affect  districts’  use  of  Eisenhower  resources. 
This  section  discusses  these  findings. 

The  level  of  poverty  in  a district  sometimes  is  significantly  associated  with  particular  patterns 
of  support  for  Eisenhower  activities.  Our  data  from  Eisenhower  coordinators  indicate  that,  compared 
to  lower-poverty  districts,  higher-poverty  districts’  portfolios  of  Eisenhower-assisted  activities: 

♦ have  more  participations  in  reform  activities  and  fewer  participations  in  traditional 
activities; 

♦ offer  more  types  of  Eisenhower-assisted  activities,  both  traditional  and  reform;  and 

♦ place  more  emphasis  on  recruiting  teachers  of  special  populations  of  students. 

The  Eisenhower  program’s  funding  formula  provides  more  funds  to  districts  that  serve  poorer 
populations,  and  our  findings  suggest  that  having  more  funds  available  enables  a district  to  support 
more  types  of  activities.  Further,  districts  with  more  funds  may  be  more  willing  to  use  resources  to 
try  reform  methods  of  professional  development,  which  may  explain  why  high-poverty  districts  have 
more  teacher  participations  in  reform  types  of  activities.  In  addition,  districts  with  more  students 
from  low-income  families  probably  place  greater  emphasis  on  recruiting  teachers  of  special 
populations  because  these  districts  are  more  likely  than  others  to  serve  students  from  these 
populations.  In  addition,  high-poverty  districts  have  more  funding  from  Title  I and  other  federal 
programs,  which  also  provide  support  and  encouragement  for  professional  development  for  teachers 
of  special  groups  of  students.  So  it  seems  that  one  of  the  intentions  of  the  legislation,  to  provide 
support  for  professional  development  to  teachers  of  children  in  high-poverty  communities,  is  at  least 
in  part  being  met.  But  there  is  a great  deal  of  room  for  improving  districts’  targeting  of  their 
professional  development  activities  to  meet  the  needs  of  teachers  of  special  populations  of  students; 
and  the  participation  of  these  teachers  in  Eisenhower-assisted  professional  development  activities. 

The  analysis  for  district  size  shows  that,  compared  to  districts  with  fewer  teachers,  the 
portfolios  of  Eisenhower-assisted  activities  in  districts  with  larger  numbers  of  teachers: 

♦ have  fewer  participations  in  traditional  types  of  professional  development  (except  in  low- 
poverty  districts); 

♦ provide  more  opportunities  for  active  learning  in  professional  development  activities; 

♦ have  activities  designed  with  more  opportunities  for  collective  participation; 
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♦ offer  activities  that  span  a longer  period  of  time; 

♦ provide  a larger  number  of  different  types  of  activities,  both  traditional  and  reform;  and 

♦ place  more  emphasis  on  recruiting  teachers  of  special  populations  of  students. 

Larger  districts  may  have  fewer  participations  in  traditional  types  of  professional 
development  activities  than  smaller  districts  because,  although  large  districts  virtually  always  offer 
traditional  activities,  they  also  are  more  likely  to  offer  reform  types  of  activities.  Small  districts, 
perhaps  because  they  have  fewer  resources  and  therefore  have  to  limit  the  number  of  activities 
that  they  offer,  support  fewer  types  of  professional  development  than  large  districts.  The  result 
is  that  smaller  districts  have  more  participations  (in  percentage  terms)  in  traditional  activities 
than  large  districts.  For  the  most  part,  the  ratio  of  participations  in  reform  to  traditional  activities 
increases  as  size  increases.  The  exception  to  this  pattern  is  low-poverty  districts,  which  do  not 
have  much  variation  in  participations  between  small  and  large  districts.  These  findings  suggest 
that  large  districts  are  able  to  offer  more  reform  activities  and  less  traditional  activities  because 
of  economies  of  scale  that  enable  them  to  offer  the  sometimes  more  expensive  reform  activities 
and  because  of  infrastructure  and  organizational  advantages.  Large  low-poverty  districts  may  be 
less  inclined  to  seek  change  and  innovation  because  on  average,  their  students  perform  better 
than  students  in  other  districts. 

Districts  with  more  teachers  also  have  higher  quality  in-district  workshops  and  institutes  than 
districts  with  fewer  teachers.  The  in-district  Eisenhower-assisted  workshops  and  institutes  offered  by 
larger  districts  provide  more  opportunities  for  active  learning  and  are  more  likely  to  be  designed  to 
foster  collective  participation.  In  addition,  larger  districts  place  more  emphasis  on  recruiting  teachers 
of  special  populations  of  students  than  do  smaller  districts. 

The  observed  positive  effects  of  size  may  be  due  in  large  part  to  the  fact  that  large  districts 
often  have  more  Eisenhower  money  to  spend  on  professional  development  than  do  small  districts. 
Although  Eisenhower  funds  available  on  a per-teacher  basis  are  comparable,  a critical  mass  of  funds 
available  in  larger  districts  may  allow  them  to  offer  a wider  range  of  types  of  professional 
development,  and  to  offer  activities  that  span  over  longer  periods  of  time.  Larger  districts  may  also 
tend  to  have  more  resources  of  other  sorts,  such  as  nearby  universities,  that  will  allow  them  to  tap 
into  a wider  range  of  professional  development  types.  In  addition,  large  districts  also  have  a more 
comprehensive,  efficient  infrastructure  for  planning  and  delivering  professional  development.  This 
may  allow  them  more  opportunities  and  resources  to  shape  and  organize  their  professional 
development  activities  to  be  more  sustained  and  intensive,  and  to  be  responsive  to  the  needs  of  whole 
schools  or  groups  of  teachers  from  a school,  rather  than  just  individual  teachers.  Consortia  seem  to 
operate  primarily  like  large  districts.  They  provide  significantly  more  opportunities  for  active 
learning  and  more  types  of  activities  than  do  small  districts.  In  addition,  in  many  cases  consortia 
provide  the  same  higher  levels  of  collective  participation  and  participation  in  reform  approaches  to 
professional  development  that  large  districts  do.  Thus,  it  appears  that  consortia,  with  similar  capacity 
and  resource  advantages,  enjoy  the  same  positive  effects  of  size  as  do  large  districts. 
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Summary  and  Conclusions 


In  this  chapter,  we  examined  how  district  portfolios  of  Eisenhower-assisted  activities  vary 
according  to  subject  area  focus,  quality — as  measured  by  the  structural  and  core  features  of  the 
activities,  strategies  for  targeting  and  recruiting  teachers  for  participation,  and  how  these  differences 
are  accounted  for  by  district  poverty  and  district  size.  We  identified  a number  of  patterns  that  have 
implications  for  the  Eisenhower  program. 

First,  we  found  that  nearly  all  districts  use  Eisenhower  funds  to  support  professional 
development  in  mathematics  and  science.  These  funds  appear  to  have  played  an  important  role  in 
supporting  content  knowledge  in  these  subject  areas  in  some  districts,  and  fostering  content 
knowledge  is  critical  for  teaching  and  learning.  Continuing  the  subject  area  focus  of  Eisenhower- 
assisted  professional  development  activities  would  allow  the  program  to  continue  its  important 
contribution  in  this  area  of  professional  development. 

Second,  we  found  that  nearly  all  districts  use  Eisenhower  funds  to  support  traditional 
approaches  to  professional  development  activities  that  are  not  “sustained”  or  “intensive,”  as  intended 
by  the  Eisenhower  legislation.  Even  among  districts  that  support  reform  types  of  professional 
development  activities  generally,  many  do  not  use  Eisenhower  funds  to  do  so.  Nevertheless,  the  fact 
that  some  districts  manage  to  use  Eisenhower  funds  almost  exclusively  for  professional  development 
activities  that  have  features  of  high  quality  indicates  that  the  program  could  perhaps  do  more  to 
increase  the  number  of  districts  using  their  Eisenhower  funds  in  optimal  ways.  Along  these  lines,  our 
findings  suggest  that  since  either  reform  or  traditional  activities  can  offer  characteristics  of  high- 
quality  professional  development,  it  is  more  important  for  districts  to  focus  on  improvements  in  the 
structure  and  substance  of  the  activities,  rather  than  the  particular  type  of  activity. 

Third,  despite  an  emphasis  on  recruiting  teachers  from  high-poverty  schools,  and  other 
teachers  of  at-risk  students,  such  teachers  appear  to  participate  in  Eisenhower-assisted  activities  in 
numbers  only  somewhat  greater  than  their  proportions  in  the  population.  If  targeting  of  professional 
development  activities  is  an  important  goal  of  the  Eisenhower  program,  districts  may  wish  to  rely 
less  on  teachers  volunteering  as  the  primary  way  teachers  come  to  participate  in  Eisenhower-assisted 
activities.  One  alternative  that  could  increase  the  impact  of  professional  development  is  targeting 
certain  schools  and  requiring  all  teachers  to  participate  in  the  same  professional  development 
activity,  although  this  may  not  be  possible  in  some  districts. 

Finally,  our  data  indicate  that  high-poverty  districts,  and  large  districts  and  consortia,  often 
are  more  able  and  willing  to  support  a diverse  and  extensive  set  of  Eisenhower-funded  activities  than 
other  districts.  A greater  amount  of  funding  goes  to  such  districts,  but  this  alone  does  not  fully 
explain  the  results.  Perhaps  larger  districts  have  greater  capacity,  and  higher  poverty  large  districts 
see  the  greater  need  to  provide  new  and  more  ambitious  forms  of  professional  development. 

Thus,  this  chapter  has  identified  areas  in  which  districts  need  to  focus  in  order  to  bring 
Eisenhower-assisted  activities  more  in  line  with  the  intent  of  the  legislation.  But  the  Eisenhower 
legislation  already  contains  provisions  that  intend  to  move  districts  toward  higher  quality 
professional  development  activities.  By  requiring  that  districts  align  Eisenhower-assisted 
professional  development  activities  with  state  and  district  standards,  coordinate  these  activities  with 
those  funded  by  other  federal  programs,  and  plan  and  evaluate  professional  development  activities, 
the  legislation  intends  to  foster  high-quality  professional  development. 


In  the  next  chapter  we  examine  these  aspects  of  the  district-level  management  and 
implementation  of  Eisenhower-assisted  professional  development  activities.  We  examine  how  school 
districts  align,  coordinate,  plan,  and  evaluate  their  activities,  and  how  district  context  (i.e.,  size  and 
poverty)  is  associated  with  these  district  operations.  In  addition,  we  present  a model  of  how  district 
management  and  implementation  is  associated  with  features  of  its  “portfolio”  of  Eisenhower-assisted 
professional  development  activities,  including  structural  and  core  features  that  are  associated  with 
improvements  in  teacher  knowledge  and  skills  and  changes  in  teacher  practice.  The  goal  of  the  next 
chapter  is  to  identify  and  describe  key  factors  in  how  districts  manage  and  implement  the  program, 
and  relate  these  to  the  district’s  use  of  Eisenhower  funds  to  support  high-quality  professional 
development. 


O 

ERIC 


* ' 4-47 


CHAPTER  5 


DISTRICT  MANAGEMENT  OF  EISENHOWER-ASSISTED 
PROFESSIONAL  DEVELOPMENT  ACTIVITIES 


This  chapter  explores  how  districts  manage  and  operate  Eisenhower-assisted  professional 
development  activities.  It  also  examines  how  key  management,  planning  and  implementation 
provisions  of  the  Eisenhower  legislation  are  associated  with  the  quality  of  professional  development 
activities,  as  defined  by  their  structural  and  core  features.  The  previous  chapter  described  the 
tremendous  variation  across  districts  in  their  portfolios  of  Eisenhower-assisted  activities — especially 
in  their  emphasis  on  traditional  vs.  reform  methods  of  professional  development,  the  duration  of  their 
activities,  and  the  extent  of  collective  participation  and  active  learning  opportunities  of  these 
activities.  We  also  examined  how  districts  select  and  target  teachers  to  participate  in  Eisenhower- 
assisted  activities.  These  features  of  district  portfolios  of  Eisenhower-assisted  activities  are,  in  part, 
the  cumulative  result  of  districts’  operation  of  Eisenhower-assisted  activities. 

The  Elementary  and  Secondary  Education  Act  (ESEA),  as  amended  by  the  Improving 
America’s  Schools  Act  (IASA)  of  1994,  contains  a number  of  provisions  about  how  districts  should 
manage  and  operate  Eisenhower-assisted  professional  development  activities.  First,  several 
provisions  of  the  law  stipulate  that  Eisenhower  funds  should  be  an  integral  part  of  state  and  district 
strategies  to  transform  education.  The  ESEA  states  that  districts  must  use  their  Eisenhower  funds  to 
support  professional  development  activities  that  are  aligned  with  challenging  state  and  local  content 
and  performance  standards.  Furthermore,  the  ESEA  requires  that  district  Eisenhower-assisted 
activities  be  coordinated  with  other  sources  of  funding  for  professional  development  activities,  as 
appropriate. 

Second,  a group  of  provisions  in  the  legislation  sets  forth  procedures  that  districts  are  to 
follow  in  order  to  achieve  its  ultimate  goals  of  improved  teacher  practice  and  student  performance. 
The  ESEA  incorporates  the  federal  government’s  emphasis  on  program  performance  and  results. 
These  procedures  are  grounded  in  a “continuous  improvement”  paradigm  that  has  permeated  all 
federal  programs  in  recent  years,  spurred  by  the  Government  Performance  and  Results  Act  of  1993 
(GPRA).  GPRA  requires  a process  of  strategic  planning  for  federal  agencies  that  includes 
developing  goals  and  measurable  objectives,  describing  how  they  would  be  achieved,  and  using 
evaluation  data  to  track  progress  toward  these  objectives.  Similarly,  the  ESEA  requires  that  states 
and  districts  assess  their  progress  in  meeting  Title  II  performance  indicators. 

A third  set  of  requirements  in  the  legislation  focuses  on  planning  Eisenhower-assisted 
professional  development  activities.  Districts  are  required  to  work  with  teachers  and  other  school- 
level  staff  in  planning  professional  development  activities.  The  law  especially  emphasizes  involving 
staff  in  Title  I schools  in  planning  Eisenhower-assisted  activities. 

The  common  underpinning  of  all  of  these  legislative  provisions  is  the  assumption  that  they 
will  improve  the  quality  of  professional  development  offered  by  school  districts.  By  including 
requirements  about  the  operation  of  district  programs,  the  legislation  attempts  to  specify  practices 
that  presumably  will  contribute  to  high-quality  professional  development. 
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In  giving  districts  a prominent  role  in  operating  and  managing  Eisenhower-assisted  activities, 
the  legislation  is  consistent  with  recent  literature  on  school  reform,  which  also  emphasizes  the  critical 
role  of  school  districts  in  setting  the  context  for  professional  development  activities  (Elmore,  1996; 
Knapp  et  al.,  1991;  Spillane,  1996;  Spillane  & Jennings,  1997).  However,  very  little  is  known  about 
how  districts  plan  or  operate  professional  development  activities,  or  about  which  district  strategies 
for  fostering  high-quality  professional  development  activities  are  effective.  Much  of  the  professional 
development  literature  focuses  on  the  optimal  characteristics  of  individual  professional  development 
activities,  and  not  on  district  strategies  for  professional  development. 

Recent  research  focuses  on  the  importance  of  school-based  professional  development  that  is 
embedded  in  the  daily  life  of  teachers  (Corcoran,  1995;  U.S.  Department  of  Education,  1999a;  Little, 
1982;  Loucks-Horsley  et  al.,  1998).  Yet,  even  when  professional  development  occurs  at  the  school, 
districts  can  play  a central  role  in  its  planning  and  implementation.  They  also  play  a role  in  building  a 
vision  of  school  reform  and  shaping  professional  development  to  support  reform  efforts  (Elmore  & 
Burney,  1996).  Districts,  for  example,  may  play  a key  role  in  conveying  to  administrators,  teachers, 
and  providers  of  professional  development  the  implications  of  state  and  local  standards  and 
assessments  for  professional  development  activities,  and  how  various  sources  of  funding  could  be 
used  to  support  these  activities.  Districts  also  may  play  a role  in  providing  technical  support  to 
schools  in  tracking  progress  toward  achieving  professional  development  goals.  Certainly  districts 
can  guide  the  use  of  Eisenhower  funds  toward  school-based  programs  that  are  coherent  (i.e., 
consistent  with  teachers’  goals,  aligned  with  state  standards  and  assessments,  and  encouraging  of 
continuing  professional  communication  among  teachers),  a key  attribute  of  quality  identified  in 
Chapter  3. 

In  the  present  chapter,  we  take  a more  in-depth  look  at  the  district’s  role  in  shaping 
Eisenhower-assisted  professional  development  activities.  Specifically,  we  examine  how  districts 
address  the  alignment  and  coordination  of  the  Eisenhower  Professional  Development  Program  with 
other  programs;  how  districts  implement  “continuous  improvement”  based  on  indicators,  needs 
assessments,  evaluation,  and  guidance  to  schools  and  providers  of  professional  development;  and 
how  districts  involve  teachers  and  other  school  staff  in  planning  professional  development  efforts. 
Finally,  we  examine  how  all  of  these  efforts  are  associated  with  the  quality  of  Eisenhower-assisted 
professional  development  in  the  district.  Exhibit  5.0  illustrates  this  chapter’s  focus  on  these  key 
aspects  of  the  districts’  operation  of  Eisenhower-activities  in  the  context  of  the  entire  study. 
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EXHIBIT  5.0 

Conceptual  Framework  for  This  Evaluation 
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Chapter  2 


Data  Sources 

The  results  presented  in  this  chapter  are  based  on  data  from  our  national  survey  of  district 
Eisenhower  coordinators  as  well  as  data  from  case  studies  of  districts  across  the  country.  We 
conducted  telephone  interviews  in  the  spring  of  1998  with  a national  random  sample  of  district 
Eisenhower  coordinators.  Through  a process  of  stratified  random  sampling,  selected  to  allow 
variation  on  size  and  poverty  level,  we  targeted  a total  of  400  districts  across  the  country.  We 
obtained  survey  data  from  a sample  of  363  district  Eisenhower  coordinators,  yielding  a response  rate 
of  88  percent.  During  the  telephone  interviews,  coordinators  reported  on  specific  professional 
development  activities  that  occurred  from  July  1997  through  December  1997;  questions  about 
general  practices  applied  to  the  1997-1998  school  year.  The  probability  of  a district  being  chosen  for 
our  sample  was  proportional  to  district  size  (i.e.,  the  number  of  teachers  in  the  district).  As  a result, 
all  of  the  data  are  weighted  by  district  size.  Therefore,  our  findings  provide  information  according  to 
the  percent  of  teachers  in  a district. 

Our  case  study  information  is  drawn  from  two  sources.  One  source  is  a series  of  10  in-depth 
case  studies  that  we  conducted  during  the  1997-1998  school  year.  The  10  case  study  districts  are  a 
purposefully  drawn  sample  of  districts,  two  from  each  of  five  states.  We  selected  sites  to  obtain 
variation  on  state-level  reform  efforts  and  the  districts’  approach  to  professional  development,  as 
well  as  demographic  and  geographic  characteristics.  We  also  draw  on  six  exploratory  case  study 
districts  that  we  visited  at  the  end  of  the  1996-1997  school  year;  we  selected  these  districts  primarily 
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for  diversity  of  region,  urbanicity,  and  ethnic  composition.  Appendix  A contains  detailed 
information  about  our  methodology  for  sampling  the  National  Profile,  and  Appendix  B contains 
detailed  information  about  our  methodology  for  selecting  case  studies.1-2 

Organization  of  Chapter 

This  chapter  is  organized  in  six  sections.  The  first  three  sections  are  organized  according  to 
the  three  main  areas  around  which  district  roles  revolve:  1)  building  a vision  for  education  reform  by 
aligning  professional  development  with  standards  and  assessments  and  coordinating  with  other 
programs;  2)  implementing  the  vision  for  professional  development  through  continuous  improvement 
based  on  the  use  of  objective  data  (through  use  of  indicators,  needs  assessments  and  evaluations  of 
Eisenhower-assisted  activities),  and  the  provision  of  guidance  to  schools  and  professional 
development  providers;  and  3)  involving  teachers  in  planning  for  professional  development. 

Throughout  these  three  sections,  for  key  management  and  implementation  variables,  we 
analyze  and  report  whether  there  are  statistically  significant  differences  according  to  district  poverty 
level  or  the  number  of  teachers  in  the  district  (i.e.,  district  size).  For  these  analyses,  poverty  is 
divided  into  three  levels — low  (less  than  10.9  percent  of  children  in  poverty),  medium  (10.9  to  21.4 
percent  of  children  in  poverty)  and  high  (greater  than  21.4  percent  of  children  in  poverty).3  District 
size  is  divided  into  four  types— small  (less  than  250  teachers),  medium  (between  250  and  1500 
teachers),  large  (more  than  1500  teachers),  and  consortia.  A consortium  is  a group  of  districts,  which 
can  range  in  size  from  only  a couple  of  districts  to  several  hundred  districts.  To  identify  consortia, 
we  asked  each  district  that  we  sampled  whether  or  not  the  district  participated  in  the  Eisenhower 
Professional  Development  Program  through  a consortium.  If  the  district  did  participate  through  a 
consortium,  we  then  drew  the  entire  consortium  into  our  sample,  and  adjusted  the  probability  of  each 
of  the  consortia  being  selected  into  the  sample,  based  on  the  full  set  of  districts  that  belonged  to  the 
consortium. 

The  size  and  poverty  effects  are  each  estimated  where  the  other  is  held  constant,  so 
significant  results  for  one  dimension  are  independent  of  the  other  dimension.  Interaction  effects 
between  size  and  poverty  are  not  statistically  significant  unless  otherwise  noted.  The  fourth  section  of 
this  chapter  provides  a summary  and  discussion  of  how  districts  vary  in  alignment,  coordination, 
continuous  improvement,  and  planning  according  to  the  district’s  poverty  level  and  size. 

The  fifth  section  of  the  chapter  presents  a model,  based  on  our  national  data  from  district 
Eisenhower  coordinators,  of  how  district  management  and  implementation  practices  influence  the 
structural  and  core  features  of  Eisenhower-assisted  professional  development  activities.  The  sixth 

The  Study  of  Educational  Resources  and  Federal  Funding  (SERFF)  collected  data  on  co-funding  of  Eisenhower 
activities  and  several  other  issues  concerning  the  resources  used  for  professional  development  in  school  districts. 

(See  Chambers,  Lieberman,  Parrish,  Kaleba,  Van  Campen,  and  Stullich,  1999.)  In  general,  SERFF  results  are 
consistent  with  those  reported  here.  Differences  in  results  and  those  reported  in  the  SERFF  are  primarily  due  to 
cross-study  differences  in  data-weighting  procedures,  the  wording  of  the  items,  or  the  presentation  of  conditional  vs. 
unconditional  results.  (For  example,  the  data  on  co-funding  of  Eisenhower-assisted  activities  from  this  evaluation  of 
the  Eisenhower  program  is  based  on  districts  where  the  program  operates,  while  the  SERFF  results  are  based  on  all 
districts.)  When  these  differences  are  taken  into  account,  the  results  of  the  two  studies  are  quite  consistent. 

Results  for  some  analyses  reported  in  this  chapter  were  reported  earlier  in  U.S.  Department  of  Education  (1999b). 
The  earlier  results  differ  from  results  in  this  report  because  they  were  preliminary,  unweighted,  and  did  not  include 
the  full  sample  of  teachers  and  districts.  Results  are  considered  to  be  statistically  significant  if  the  p- value  is  .05  or 
smaller. 

These  categories  divide  the  population  equally  into  thirds. 
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and  final  section  summarizes  our  major  findings  and  discusses  implications  for  both  federal  and 
district  policy. 


Building  a Vision  for  Professional  Development: 
Alignment  with  Standards  and  Assessments,  and 
Coordination  with  Other  Programs 


Section  Findings 

$ Most  of  the  nation  s teachers  are  in  districts  where  Eisenhower  coordinators  report 
substantial  alignment  of  Eisenhower-assisted  activities  with  state  and  district  standards , 
and  to  a lesser  extent , with  state  and  district  assessments. 

♦ Many  of  the  nation ’s  teachers  are  in  districts  where  coordinators  report  that  other 
district  staff,  in  addition  to  the  Eisenhower  coordinator ; are  involved  in  planning 
Eisenhower-assisted  activities,  and  that  district  Eisenhower  coordinators  work  together 
with  other  district-level  program  administrators,  especially  mathematics  and  science 
curriculum  specialists. 

$ There  is  more  coordination  of  the  Eisenhower  program  with  other  mathematics  and 
science-oriented  initiatives,  in  comparison  with  initiatives  that  do  not  focus  on  these 
subjects.  In  districts  where  other  federal  programs  operate,  most  teachers  are  in  districts 
where  Eisenhower  coordinators  report  working  with  administrators  of  other  federal 
education  programs,  especially  those  funded  by  NSF.  Eisenhower  coordinators  to  a 
lesser  extent  report  working  with  administrators  of  ED -funded  programs,  such  as  Title  I, 
Part  A. 

§ Across  all  programs,  districts  are  substantially  less  likely  to  report  co-funding  activities 
than  working  with  other  staff;  co-funding  of  activities  is  more  common  with  NSF-funded 
activities  than  with  those  funded  by  ED  programs.  However,  we  have  no  national  data 
regarding  the  actual  proportion  of  Eisenhower-assisted  activities  or  participations  that 
are  co-funded  with  other  programs. 

❖ There  is  more  coordination  in  larger  and,  to  some  extent,  higher-poverty  districts,  than  in 
other  districts. 

The  Eisenhower  program  predates  recent  moves  toward  systemic  education  reform,  and  it 
operates  as  just  one  of  a number  of  funding  streams  for  professional  development  in  the  nation’s 
school  districts.  In  some,  generally  small  districts,  Eisenhower  funds  may  exist  alongside  a few  other 
education  programs,  such  as  Title  I,  Part  A;  in  other,  usually  large  districts,  it  is  one  of  a panoply  of 
federal,  state,  and  local  programs,  reform  efforts,  and  professional  development  initiatives.  The 
provisions  of  the  Eisenhower  legislation  give  prominence  to  the  district’s  role  in  integrating 
Eisenhower-assisted  professional  development  activities  into  other  state  and  district  education  reform 
efforts  through  alignment  with  challenging  education  standards,  and  coordination  with  other 
education  programs. 


(.■ 


5-5 


The  Title  II  legislation  reflects  the  importance  of  aligning  professional  development  with 
challenging  state  and  local  standards.  Specifically,  the  legislation  stipulates  that  the  LEA’s  plan  shall 
“be  aligned  with  the  State's  challenging  State  content  standards  and  challenging  State  student 
performance  standards”  (Section  2208  (d)(1)(C))  and  "describe  a strategy,  tied  to  challenging  State 
content  standards  and  challenging  State  student  performance  standards,  consistent  with  the  needs 
assessment  under  subsection  (b)”  (Section  2208  (d)(1)(D)).  Title  I of  ESEA  requires  that  by  the 
1997-98  school  year,  each  state  was  to  have  adopted  challenging  content  standards,  in  at  least  reading 
and  mathematics,  and  challenging  performance  standards  that  describe  students’  mastery  of  the 
content  standards.  Forty-eight  states,  plus  Puerto  Rico  and  the  District  of  Columbia,  have  met 
federal  requirements  for  developing  challenging  statewide  content  standards.  Twenty-one  states, 
plus  Puerto  Rico,  have  met  the  requirement  for  developing  student  performance  standards  (U.S. 
Department  of  Education,  1999b). 

Aligning  professional  development  with  standards  and  assessments  can  be  one  way  that 
districts  work  to  send  a common  message  to  teachers  about  appropriate  instruction  (Webb,  1998). 
Lessons  from  systemic  reform  indicate  that  fragmentation  within  the  system  decreases  motivation  for 
working  on  reform  (Fullan,  1993,  1996),  and  that  success  is  in  part  contingent  upon  establishing 
long-term  goals  and  being  able  to  articulate  a new  vision  (Cohen  & Spillane,  1992).  When  policies 
and  reforms  are  unaligned,  teachers  notice  the  inconsistency  (Grant,  Peterson,  & Shojgreen-Downer, 
1996);  alternatively,  when  policies  and  reforms  are  aligned,  this  can  work  to  encourage  changes  in 
instruction  (Spillane  & Jennings,  1997). 

In  the  literature,  there  is  little  evidence  that  districts  generally  provide  a common  vision  that 
would  guide  the  use  of  professional  development  funds  from  a variety  of  sources,  or  link  professional 
development  to  other  education  reform  efforts.  In  addition,  research  indicates  that  districts  generally 
are  unaccustomed  to  planning  portfolios  of  professional  development  activities  strategically  to 
achieve  instructional  goals  or  other  types  of  goals  and  objectives  (Elmore,  1993).  Furthermore, 
professional  development  generally  does  not  appear  to  be  embedded  in  the  daily  activities  of  schools 
and  teachers.  Rather,  “most  school  systems  see  professional  development  as  a discrete  activity. . . or 
service  that  is  provided  to  schools  as  one  of  a number  of  centrally  organized  administrative 
functions”  (Elmore  & Burney,  1996,  p.  23). 

The  consequence  typically  is  a menu  of  discrete  professional  development  activities,  usually 
focused  on  specific  content  areas  or  pressing  issues  in  the  daily  conduct  of  schooling,  such  as  school 
discipline.  As  a result,  activities  are  often  organized  and  delivered  centrally  so  that  school  personnel 
participate  in  training  that  is  designed  and  conducted  in  isolation  from  their  work  setting  (Elmore  & 
Burney,  1996). 

In  their  seminal  case  study  of  professional  development  in  New  York  City’s  Community 
School  District  2,  Elmore  and  Burney  (1996)  emphasize  the  critical  importance  of  establishing  a 
focus  for  guiding  professional  development  activities  in  the  district.  They  describe  a district  that 
systematically  identified  instructional  goals  and  objectives  and  designed  a strategy  for  professional 
development  to  address  directly  these  goals  and  objectives.  The  district’s  emphasis  on  instructional 
improvement  focused  its  professional  development  activities.  They  view  this  district  as  an  “existence 
proof’  that  districts  can  “be  agents  of  serious  instructional  improvement,”  and  that  districts  can  use 
professional  development  as  a tool  for  the  reform  of  schools.  Other  researchers  also  have 
emphasized  the  importance  of  establishing  a district  vision  for  professional  development  (Spillane, 
1996)  and  the  important  role  of  standards  and  assessments  in  shaping  professional  development 
activities  (Cohen  & Hill,  1998). 
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Besides  alignment,  another  way  for  districts  to  focus  professional  development  activities  is 
by  coordinating  multiple  sources  of  funding  for  professional  development  activities.  The  ESEA 
supports  the  idea  of  coordinating  funding  streams.  The  legislation  indicates  that  Eisenhower  funds 
should  not  be  spent  in  isolation  from  other  program  funds,  but  instead  Eisenhower-assisted  activities 
should  be  planned  and  coordinated  with  other  sources  of  funding  for  professional  development.  In 
addition,  the  law  requires  that  state  and  local  plans  describe  how  Eisenhower-assisted  activities  are 
coordinated  with  other  Education  Department  (ED)  programs  (such  as  Title  I,  Part  A of  ESEA  and 
the  Individuals  with  Disabilities  Education  Act  (IDEA)),  as  well  as  with  professional  development 
efforts  supported  with  funds  they  receive  from  other  federal  agencies  (such  as  the  National  Science 
Foundation  ) (Section  2205(c)  and  Section  2208(d)(1)(H)). 

Districts  often  must  balance  many  sources  of  funding  for  professional  development.  When 
districts  have  established  a focus  for  their  professional  development,  they  may  deploy  these  multiple 
sources  of  funding  toward  the  same  ends.  Leveraging  resources  can  have  a role  in  affecting  the 
quality  of  professional  development  (Corcoran,  1995).  For  example,  Elmore  and  Burney  (1996) 
describe  the  “multi-pocket  budgeting”  that  occurred  in  District  2,  where  the  district  administration 
used  funding  from  multiple  sources  to  fund  its  coherent  professional  development  strategy. 

In  this  section  of  the  chapter,  we  use  our  district  coordinator  survey  data  to  examine  the 
extent  to  which  districts  integrate  Eisenhower-assisted  activities  with  other  education  reform  efforts, 
which  we  measure  by  the  districts’  reported  degree  of  alignment  and  coordination.  We  first  describe 
the  extent  to  which  the  district  coordinators  report  linking  professional  development  activities  to  state 
and  district  standards  and  assessments.  We  then  examine  the  level  of  coordination  between 
Eisenhower-funded  activities  and  other  programs.  Coordination  is  measured  by  the  extent  of 
collaboration  with  others  in  professional  development  roles  and  with  other  federal  programs,  and  by 
the  extent  to  which  districts  use  funding  streams  in  a strategic  way  by  combining  funding  sources. 
Taken  together,  these  aspects  of  alignment  and  coordination  can  be  thought  of  as  an  indication  of  the 
extent  to  which  the  district  is  providing  a coherent  vision  for  professional  development. 

Alignment  of  Eisenhower-assisted  Activities  with  State  and  District  Standards 
and  Assessments 

“Alignment”  is  a difficult  concept  to  define  and  measure.  Ideally,  all  aspects  of  an 
educational  system  are  aligned  with  each  other  to  support  student  learning  to  high  standards.  This 
might  mean  that  textbooks,  other  materials,  and  instructional  approaches  that  teachers  use  match  state 
and  local  standards  and  assessments,  that  the  professional  development  teachers  receive  helps  them 
to  use  those  materials  and  approaches  appropriately,  and  that  only  what  is  valued  is  included  in  the 
curriculum  or  in  assessments. 

Evidence  of  alignment  can  be  sought  in  many  places.  Webb  (1998)  notes  that  one  could  look 
for  alignment  in  the  consistency  of  content  focus  between  professional  development  activities  and 
standards  and  assessments;  the  extent  to  which  professional  development  activities  and  standards  and 
assessments  are  rooted  in  a common  view  of  how  students  learn;  and  the  extent  to  which  professional 
development  and  standards  and  assessments  reflect  the  view  that  all  students  must  learn  to  high 
standards,  which  indicates  attention  to  the  learning  needs  of  diverse  students. 

To  measure  how  well  districts  are  doing  in  meeting  the  legislative  requirements  in  this  area, 
we  asked  district  Eisenhower  coordinators  participating  in  our  national  survey  about  the  existence  of 
state  and  local  standards  and  assessments  in  mathematics  and  science,  and  the  extent  to  which 
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Eisenhower-assisted  activities  are  intended  to  support  those  standards  and  assessments.  Specifically, 
we  asked  district  Eisenhower  coordinators  whether  statewide  or  district-wide  standards  or  curriculum 
frameworks,  or  assessments,  in  mathematics  or  science,  have  been  adopted.  We  then  asked  them  to 
what  extent  the  Eisenhower-assisted  activities  in  their  district  were  designed  to  help  teachers  adapt 
their  teaching  to  meet  the  particular  standards  or  assessments.  Response  categories  were:  1)  the 
activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  standards  or 
assessments,  2)  the  activities  are  designed  to  some  extent  to  do  this,  or  3)  the  activities  are  designed 
to  a large  extent  to  do  this. 

As  Exhibit  5.1a  shows,  Eisenhower  coordinators  indicate  that  their  states  typically  have 
standards  and  assessments  in  mathematics  and  science.  Ninety-one  percent  of  teachers  are  in  states 
with  mathematics  standards,  and  85  percent  of  teachers  are  in  states  with  science  standards, 
according  to  district  Eisenhower  coordinators. 

EXHIBIT  5.1a 

Percent  of  Teachers  in  Districts  Where  State  and  District  Mathematics  and  Science 

Standards  and  Assessments  Exist  (n=  363) 
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Source.  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  9 1 percent  of  teachers  are  in  districts  that  have  state  mathematics  standards.  Each  bar  and  the 
number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 


In  addition,  Exhibit  5.1a  shows  that  most  Eisenhower  coordinators  also  report  the  existence 
of  state  assessments,  though  slightly  less  frequently  for  science;  91  percent  of  teachers  are  in  districts 
that  report  having  state-level  mathematics  assessments,  while  only  72  percent  of  teachers  are  in 
districts  that  say  they  have  statewide  science  assessments.  Districts  have  their  own  mathematics  and 
science  standards  somewhat  less  frequently  than  states  in  which  they  are  located.  Eighty-five  percent 
of  teachers  are  in  districts  that  report  that  they  have  district  standards  in  mathematics,  and  78  percent 
are  in  districts  that  have  district  science  standards.  Further,  districts  are  considerably  less  likely  to 
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have  local-level  assessments,  particularly  in  science;  69  percent  of  teachers  are  in  districts  that  have 
district  assessments  in  mathematics,  and  42  percent  of  teachers  are  in  districts  that  have  district 
assessments  in  science. 

Exhibit  5.1b  shows  that,  of  the  districts  that  report  that  they  have  state  and/or  district 
standards,  83  to  86  percent  of  teachers  are  in  districts  that  say  that  Eisenhower-assisted  activities  in 
mathematics  and  science  are  designed  to  support  these  standards  “to  a large  extent.”  Fewer 
coordinators  report  such  support  for  assessments;  only  67  to  72  percent  of  teachers  are  in  districts 
where  Eisenhower  activities  in  mathematics  and  science  support  state  and  local  assessments  “to  a 
large  extent.”  However,  the  vast  majority  of  teachers  (94  percent  or  more)  are  in  districts  where 
Eisenhower-assisted  activities  support  state  and  local  assessments  either  “to  some  extent”  or  “to  a 
large  extent”  (data  not  shown). 


EXHIBIT  5.1b 

Percent  of  Teachers  in  Districts  Where  Eisenhower-assisted  Activities  Are  Aligned 
“to  a Large  Extent”  with  State  and/or  District  Standards  and/or  Assessments  (Where 
Such  Standards  and  Assessments  Exist)  (n  varies) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1 998. 

How  to  read  this  exhibit : The  first  bar  shows  that  of  the  teachers  in  districts  with  state  mathematics  standards,  85  percent  of  teachers  are  in 
districts  reporting  that  their  Eisenhower-assisted  professional  development  activities  are  designed  “to  a large  extent”  to  help  teachers  adapt  to  the 
state  mathematics  standards.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 


There  are  a number  of  reasons  that  Eisenhower  activities  may  be  less  aligned  with 
assessments  than  with  standards.  First,  the  assessments  themselves  may  not  yet  be  appropriately 
aligned  to  the  standards.  This  may  be  true,  in  part,  because  the  ESEA  requirement  for  assessments  to 
be  aligned  with  standards  had  not  yet  gone  into  effect  at  the  time  that  we  collected  our  data.4 
Furthermore,  developing  assessments,  especially  those  that  are  aligned  with  high  standards,  has 
proven  to  be  harder  than  developing  the  standards  themselves. 


4 While  Title  I of  ESEA  required  that  each  state  adopt  challenging  content  and  performance  standards  in  at  least 
reading/language  arts  and  mathematics  by  the  1997-98  school  year,  final  assessment  systems  are  not  required  to  be 
in  place  until  2000-2001  (U.S.  Department  of  Education,  1999c). 
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Our  data  also  show  that  where  district  standards  exist,  Eisenhower  activities  are  as  likely  to 
. be  aligned  with  than  as  with  state  standards;  similarly,  where  district  assessments  exist,  Eisenhower- 
assisted  professional  development  activities  are  equally  likely  to  be  aligned  with  state  and  district 
assessments.  This  is  probably  because  states  generally  have  more  responsibility  than  districts  for 
establishing  and  implementing  standards;  in  addition,  district  standards  often  mirror  state  standards. 

To  determine  whether  the  degree  of  alignment  between  Eisenhower-assisted  professional 
development  activities  and  standards  and  assessments  differs  according  to  district  poverty  or  district 
size,5  we  developed  a composite  measure  of  alignment.  It  is  a standardized  scale,  for  which  five 
indicates  that  district  coordinators  report  professional  development  activities  to  be  aligned  “to  a large 
extent”  with  both  state  and  district  standards  and  assessments,  and  zero  indicates  that  district 
coordinators  report  not  being  aligned  with  standards  or  assessments  at  either  level.  The  value  of  the 
scale  for  each  district  is  based  on  the  degree  to  which  the  district  reports  being  aligned  with  whatever 
state  and  district  standards  and  assessments  exist  in  the  district.  As  Exhibit  5.1c  indicates, 
Eisenhower-assisted  activities  in  large  districts  are  significantly  more  likely  to  be  aligned  with  state 
and  district  standards  and  assessments  than  Eisenhower-assisted  activities  in  small  districts.  This 
may  be  because  large  districts  have  a more  developed  and  sophisticated  method  for  integrating  state 
and  district  reforms  with  professional  development  activities.  Previous  research  has  shown  that 
administrators  often  believe  that  they  need  more  information  on  how  to  link  professional 
development  with  standards  (Celebuski  & Farris,  1998),  and  this  lack  of  information  may  be  more 
prevalent  in  small  than  large  districts.  It  also  may  be  that  districts  with  more  teachers  have  more  of  a 
need  to  create  an  organized  strategy  for  the  design  of  their  professional  development  activities,  and 
thus  are  more  likely  than  smaller  districts  to  use  state  and  district  standards  and  assessments  for  this 
purpose.  Poverty-level  differences  in  alignment  are  not  significant. 

Case  studies  help  to  explain  just  how  important  state  and  local  standards  and  assessments  can 
be  in  shaping  Eisenhower-assisted  professional  development  activities.  For  example,  in  our  two  case 
districts  in  Texas,  Eisenhower  coordinators  and  other  district  administrators  frequently  referred  to  the 
Texas  Assessment  of  Academic  Skills  (TAAS),  the  statewide  assessment,  as  a critical  determinant  of 
the  content  of  professional  development.  Indeed,  in  one  of  the  two  districts,  administrators  told  us 
that  everything  they  do  is  guided  by  TAAS.  They  report  that  professional  development  activities  are 
geared  to  areas  of  the  TAAS  on  which  the  district  has  identified  a need  for  improvement.  To  a lesser 
extent,  district  administrators  indicate  that  the  Texas  Essential  Knowledge  and  Skills  (TEKS),  the 
state’s  new  content  standards,  also  are  an  important  determinant  of  professional  development.  Texas 
illustrates  that  while,  on  average,  standards  are  more  influential  than  assessments,  this  is  not  always 
the  case.  In  Texas,  state  assessment  results  are  used  for  important  accountability  purposes,  and  this 
probably  explains  their  greater-than-average  influence. 


5 As  we  noted  earlier,  district  poverty  and  district  size  are  always  estimated  together  in  the  same  model,  so  any 
significant  effects  for  size  control  for  poverty,  and  likewise  any  significant  effects  for  poverty  control  for  size. 
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EXHIBIT  5.1c 


Degree  of  Alignment  Between  Eisenhower-assisted  Activities  and  Standards  and 
Assessments,  Overall  and  by  District  Poverty  and  District  Size  (n=363) 
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Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  are  in  districts  that  have  an  average  alignment  of  4.0 
on  a scale  from  zero  to  five,  where  zero  indicates  no  alignment  and  five  indicates  being  aligned  “to  a large  extent”  with  state  and 
district  standards  and  assessments.  The  degree  of  alignment  differs  significantly  by  district  size,  but  not  by  district  poverty  level.  Each 
dot  represents  one  district.  As  the  number  districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases 
in  length.  Each  distribution  represents  the  distribution  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the 
mean. 


Our  two  case  districts  in  Kentucky  place  a similar  emphasis  on  providing  professional 
development  aligned  with  the  Kentucky  Instructional  Results  Information  System  (KIRIS),  the  state 
student  assessment  program  used  in  their  school  accountability  program.  In  both  districts,  this 
alignment  takes  the  form  of  the  district  or  schools  selecting  professional  development  activities 
designed  to  address  areas  of  low  performance  on  KIRIS.  District  administrators  in  both  the  Texas 
and  the  Kentucky  districts  repeatedly  refer  to  their  statewide  assessments  systems  as  important 
factors  in  all  decisions  made  about  professional  development. 

Despite  the  central  role  that  state  assessments  often  play  in  the  design  of  professional 
development,  this  type  of  alignment  does  not  necessarily  contribute  to  a consistent  approach  to 
professional  development.  For  example,  one  Kentucky  district  selects  professional  development 
activities  in  response  to  areas  of  weak  performance  by  students  on  the  previous  year's  state 
assessments.  Thus,  plans  for  professional  development  could  change  on  an  annual  basis.  During  our 
site  visit,  we  were  told  that  all  professional  development  activities  would  shift  from  mathematics  to 
science,  in  response  to  that  year’s  test  scores.  To  the  extent  that  areas  of  weakness  in  student 
achievement  change  from  year  to  year,  which  is  not  unusual  according  to  district  officials,  this 
approach  may  not  support  the  kind  of  sustained  professional  development  efforts  envisioned  by  the 
Eisenhower  legislation. 

Both  our  quantitative  and  qualitative  data  illustrate  the  emphasis  that  many  districts  place  on 
aligning  professional  development  activities  with  state  and  district  standards  and  assessments. 
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However,  neither  the  survey  nor  the  case  study  data  allow  us  to  evaluate  all  of  the  facets  of  alignment 
discussed  by  Webb  (1998).  For  example,  we  did  not  observe  professional  development  activities. 
Direct  observation  which  would  have  enabled  us  to  determine  whether  the  content  of  these  activities 
reflects  the  depth  of  content  knowledge  demanded  by  high  standards  for  student  performance. 

One  way  that  some  districts  align  their  professional  development  with  state  standards  is  to 
make  the  process  of  achieving  such  alignment  a professional  development  activity.  Several  districts 
that  we  visited  use  Eisenhower  funds  to  support  a review  of  district  curricula  to  ensure  that  they  are 
aligned  with  state  standards.  At  least  three  case-study  districts,  one  each  in  Washington,  Texas,  and 
New  York,  use  Eisenhower  funds  in  this  way.  In  these  districts,  groups  of  teachers,  typically 
volunteers,  meet  over  an  extended  period  of  time  to  review  state  standards  and  current  district 
curricula  to  identify  areas  of  alignment  and  areas  in  which  alignment  should  be  improved.  In  all 
three  districts,  this  process  is  ongoing.  Clearly,  this  type  of  professional  development  activity  is 
aligned  to  state  standards  in  the  sense  that  it  requires  teachers  to  be  well  informed  about  what  the 
state  standards  are  and  how  they  might  translate  into  local  curricula. 

Coordination  with  Other  Programs 

Alignment  is  one  measure  of  how  Eisenhower  professional  development  activities  fit  into 
districts’  reform  efforts.  The  extent  to  which  districts  work  with  and  co-fund  activities  with  other 
programs  is  another  important  measure  of  the  integration  of  reform  efforts  in  the  district. 

Eisenhower  funds  support  professional  development  activities,  but  so  do  funds  from  other  sources — 
local,  state,  and  federal.  Eisenhower-assisted  activities  can  be  integrated  with  professional 
development  activities  funded  by  other  sources,  they  can  operate  in  isolation  from  these  other 
sources,  or  they  can  operate  independently  but  be  part  of  a larger  professional  development  plan. 
Coordination  between  Eisenhower  coordinators  and  coordinators  of  other  federal  and  district 
programs,  and  co-funding  activities  with  those  of  other  programs,  can  signal  district  attempts  to  build 
a coherent  vision  of  professional  development,  or  to  establish  a focus  for  potentially  disparate 
professional  development  activities. 

Because  the  relationship  between  Eisenhower-assisted  activities  and  professional 
development  activities  funded  through  other  sources  conceivably  can  take  many  different  forms, 
working  relationships  between  the  staff  of  the  different  programs  may  govern  the  relationships 
between  programs.  Thus,  this  section  examines  the  extent  to  which  Eisenhower  coordinators  work 
with  others  in  their  districts,  and  the  extent  to  which  Eisenhower- assisted  activities  are  co-funded 
with  activities  funded  by  other  federal  programs. 

Eisenhower  Coordinators’  Relationships  with  Others  within  the  District  Office 

We  asked  district  Eisenhower  coordinators  to  answer  questions  regarding  the  structure  of 
their  district  office,  specifically  to  identify  the  positions  present  in  their  district,  to  identify  which 
roles  they  play  in  their  districts,  and  to  indicate  whether  they  work  with  people  in  other  positions  in 
making  decisions  about  the  Eisenhower  program.  As  Exhibit  5.2  illustrates,  district  Eisenhower 
coordinators  report  that  they  do  not  work  in  isolation.  Rather,  they  report  working  with  other  district 
administrators  and  with  coordinators  of  other  federal  programs  to  make  decisions  about  Eisenhower- 
assisted  activities. 
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Exhibit  5.2  reflects  several  findings  related  to  the  nature  of  coordination.  First,  it  shows  that 
Eisenhower  coordinators  often  serve  in  multiple  roles.  Among  teachers  in  districts  that  have  a 
curriculum  coordinator,  mathematics  coordinator,  or  science  coordinator  position,  close  to  half  are  in 
districts  in  which  the  Eisenhower  coordinator  fills  these  positions  (48,  46,  and  47  percent  of  teachers 
in  districts,  respectively).  And  among  teachers  in  districts  with  a Federal  Program  Coordinator,  43 
percent  are  in  districts  in  which  the  Eisenhower  coordinator  fills  the  position.  On  average, 
Eisenhower  coordinators  report  spending  23  percent  of  their  time  administering  the  Eisenhower 
program,  and,  though  some  work  full-time  on  Eisenhower,  three-quarters  spend  27  percent  or  less  of 
their  time  on  Eisenhower  (results  not  shown).  The  fact  that  Eisenhower  coordinators  serve  in 
multiple  roles  suggests  a certain  amount  of  integration  between  Eisenhower-assisted  efforts  and  other 
district  efforts. 

Second,  Exhibit  5.2  suggests  that  when  Eisenhower  coordinators  do  not  fill  multiple  roles, 
they  report  working  closely  with  other  district-level  administrators  in  making  decisions  about  how  to 
use  Eisenhower  funds.  Almost  all  teachers  are  in  districts  where  Eisenhower  coordinators  report 
working  closely  with  mathematics  and  science  curriculum  specialists  (96  and  97  percent  of  teachers 
in  districts,  respectively).  More  than  80  percent  of  teachers  are  in  districts  whose  Eisenhower 
coordinator  reports  working  with  the  general  curriculum/instruction  coordinator  (90  percent),  the 
federal  programs  coordinator  (83  percent),  and  the  professional  development  coordinator  (81 
percent).  Seventy-eight  percent  of  teachers  are  in  districts  whose  Eisenhower  coordinator  says  they 
work  with  the  Title  I coordinator.  Teachers  are  least  likely  to  be  in  districts  where  the  Eisenhower 
coordinator  reports  working  with  special  education  coordinators  (62  percent). 

Coordination  of  Eisenhower-assisted  Activities  with  Those  of  Other  Federal 

Programs 

In  addition  to  reporting  that  they  work  closely  with  others  in  professional  development- 
related  roles,  Eisenhower  coordinators  report  that  they  work  closely  with  the  coordinators  of  federal 
education  programs.  We  provided  district  coordinators  with  a list  of  NSF  and  ED  programs,  and 
asked  them  to  indicate  whether  the  program  operated  in  their  state  or  district,  whether  it  supported 
professional  development  in  their  district  in  the  lastyear  (1996-1997),  whether  it  co-funded 
professional  development  with  Eisenhower,  and/or  if  program  staff  worked  closely  with  Eisenhower 
program  staff. 

As  Exhibit  5.3a  shows,  where  particular  federal  programs  support  professional  development 
in  their  district,  the  Eisenhower  coordinators  report  working  closely  with  staff  of  the  other  federal 
programs.  Although  a relatively  small  proportion  of  teachers  are  in  districts  that  receive  funds  from 
NSF  programs,  Eisenhower  coordinators  in  these  districts  almost  always  report  that  they  work 
closely  with  the  administrators  responsible  for  these  programs.  Ninety-one  percent  of  teachers  are  in 
districts  where  Eisenhower  coordinators  report  working  with  staff  of  the  Local  Systemic  Change 
(LSC)  initiative,  86  percent  of  teachers  are  in  districts  that  report  working  with  the  staff  of  the  Urban 
Systemic  Initiative  (USI),  and  85  percent  of  teachers  are  in  districts  that  report  working  with  the  staff 
of  the  Rural  Systemic  Initiative  (RSI),  in  districts  where  these  NSF  programs  operate.  In  states  with 
NSF  State  Systemic  Initiatives  (SSI),  59  percent  of  teachers  are  in  districts  in  which  Eisenhower 
coordinators  report  working  closely  with  SSI  staff. 


EXHIBIT  5.3a 


Percent  of  Teachers  in  Districts  in  which  Eisenhower  Activities  Coordinate 
(Co-fund  and/or  Work  Closely)  with  Other  Programs  (n  varies) 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that,  of  teachers  in  districts  with  SSIs,  66  percent  of  teachers  are  in  districts  that  co-fund  with  SSIs, 
and  59  percent  of  teachers  are  in  districts  in  which  the  district  Eisenhower  coordinator  reports  working  closely  with  the  SSI.  Co-funding  differs 
significantly  by  both  district  poverty  level  and  district  size.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for 
each  category. 

Note:  Results  on  co-funding  for  each  listed  program  are  based  on  districts  that  participate  in  the  program,  and  in  which  the  program  supports 
professional  development.  Results  for  working  closely  with  each  program  are  based  on  districts  that  participate  in  the  program. 


In  addition,  Exhibit  5.3a  shows  that  large  proportions  of  teachers  also  are  in  districts  where 
Eisenhower  coordinators  report  working  closely  with  coordinators  of  several  federal  programs 
funded  by  the  Education  Department.  Notably,  of  teachers  in  districts  with  Title  I,  Part  A programs 
(representing  almost  all  teachers),  82  percent  are  in  districts  where  Eisenhower  coordinators  report 
working  closely  with  Title  I coordinators.  A similar  proportion  of  teachers — 78  percent — are  in 
districts  where  Eisenhower  coordinators  report  working  closely  with  the  coordinators  of  Title  VI- 
funded  activities.  Of  the  nation’s  teachers  in  districts  where  other  ED  programs  operate,  between  60 
and  70  percent  are  in  districts  that  work  closely  with  staff  of  these  ED  programs,  with  the  exception 
of  the  Title  Eli  (73  percent)  and  Title  IX  (42  percent)  programs. 

In  fact,  Eisenhower  coordinators  report  working  with  multiple  federal  education  programs, 
where  these  programs  exist  in  their  districts.  Our  analyses  of  these  data  show  that  Eisenhower 
coordinators  report  that  they  work  closely  with  coordinators  of  65  percent  of  the  other  federal 
programs  that  operate  in  their  districts  (data  not  shown).  In  other  words,  in  a typical  district  with  six 
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federal  programs  other  than  Eisenhower,  the  Eisenhower  coordinator  could  be  expected  to  work 
closely  with  the  individual(s)  responsible  for  coordinating  about  four  of  those  programs.6 

Co-funding  with  Other  Programs 

Co-funding — the  contribution  of  funds  from  two  programs  to  support  the  same  professional 
development  activity — is  another,  perhaps  stronger  indicator  of  coordination  between  Eisenhower- 
assisted  activities  and  other  federal  programs.  In  addition  to  providing  information  about  the  other 
district  personnel  with  whom  the  Eisenhower  coordinator  works,  Exhibit  5.3a  shows  the  percent  of 
teachers  in  districts  in  which  Eisenhower  activities  are  co-funded  with  NSF  and  ED  programs;  the 
exhibit  demonstrates  that,  in  districts  with  a specific  program,  a substantial  proportion  of  teachers  are 
in  districts  where  Eisenhower  coordinators  report  that  they  co-fund  professional  development 
activities  with  that  program. 

With  only  a few  exceptions,  Eisenhower  coordinators  are  less  likely  to  report  co-funding 
activities  with  other  federal  programs  than  they  are  to  report  working  closely  with  those  programs. 
This  is  clear  from  Exhibit  5.3a.  Exhibit  5.3a  also  indicates  that  the  Eisenhower  program  is  more 
likely  to  co-fund  with  NSF-funded  programs  than  ED-funded  programs.  For  example,  between  66 
and  86  percent  of  teachers  are  in  districts  in  which  Eisenhower  coordinators  co-fund  with  three  of  the 
four  NSF  initiatives,  whereas  the  maximum  proportion  of  teachers  in  districts  that  co-fund  with  ED- 
sponsored  programs  is  50  percent  (for  Title  I,  Part  A).  One  explanation  of  this  pattern  is  NSF’s 
exclusive  focus  on  mathematics  and  science;  since  this  also  is  the  predominant  focus  of  Eisenhower- 
assisted  activities,  there  may  be  more  opportunity  for  co-funding  with  NSF  activities.  Also,  NSF  has 
co-funding  requirements  for  many  of  its  programs,  which  may  in  part  account  for  the  higher  levels  of 
co-funding  with  Eisenhower.  Overall,  teachers  are  in  districts  that  report  that  they  co-fund 
professional  development  activities  with  about  one-third — 34  percent — of  the  other  federal  programs 
that  support  professional  development  in  their  district  (results  not  shown). 

Adding  the  total  number  of  federal  programs  that  co-fund  with  Eisenhower,  of  a possible  10, 
allows  us  to  form  a composite  measure  of  co-funding  to  test  poverty  and  size  effects.  The  results, 
shown  in  Exhibit  5.3b,  indicate  that  on  average  districts  co-fund  with  two  programs,  and  that  district 
size  and  poverty  interactions  significantly  affect  co-funding.  Exhibit  5.3c  shows  the  interactions 
between  district  poverty  and  size  for  co-funding.  Generally,  co-funding  increases  as  district  size  and 
poverty  level  increases.  The  exception  is  that  for  consortia,  medium-poverty  districts  co-fund  less 
than  low-poverty  districts. 


6 It  is  important  to  note  that  these  are  not  necessarily  four  individuals,  one  coordinating  each  program,  but  may  be 
only  one  or  two  individuals,  each  of  whom  coordinates  more  than  one  other  program.  Similarly,  the  Eisenhower 
coordinator  him/herself  may  be  responsible  for  overseeing  other  Federal  programs. 
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EXHIBIT  5.3b 


Extent  of  Co-Funding  of  Eisenhower-assisted  Activities  with  Those  of  Other  Federal 
Programs,  Overall  and  by  District  Poverty  and  District  Size  (n=363) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  teachers  are  in  districts  that  co-fund  with  two  federal  programs. 
Each  dot  represents  one  district.  As  the  number  of  districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line 
increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular  category.  The  number  to  the  right  of  the 
distribution  is  the  mean. 


These  results  might  be  explained  by  the  fact  that  high-poverty  districts  receive  more  funding 
that  may  be  used  for  professional  development  from  federal  and  some  state  sources  whose  formulas 
target  districts  with  greater  needs.  The  existence  of  multiple  sources  of  funding  with  similar 
programmatic  goals  may  create  greater  opportunities  for  coordination.  Similarly,  large  districts  and 
consortia  may  have  funds  from  multiple  sources  which  increase  opportunities  for  coordination  and 
co-funding.  Large  districts  and  consortia  are  also  more  likely  to  have  more  individuals  in 
professional-development  related  roles,  which  may  foster  collaboration  that  results  in  funding 
activities  with  a combination  of  sources. 
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EXHIBIT  5.3c 


Extent  of  Co-funding  of  Eisenhower-assisted  Activities  with  Those  of  Other  Federal 
Programs,  Interaction  of  District  Poverty  and  District  Size  (n=363) 
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-o—  Medium  Poverty 
—a—  Low  Poverty 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  data  point  designated  by  the  first  square  indicates  that  the  average  percent  of  participations  in  reform  activities  in 
medium-poverty  small  districts  is  19  percent.  The  line  with  data  points  designated  by  diamonds  indicates  the  percent  of  participations  in  reform 
activities  for  high-poverty  districts  in  each  of  four  sizes/types  of  districts  (i.e.,  consortia,  large,  medium,  and  small  districts);  the  line  with  data 
points  designed  by  squares  indicates  the  percent  of  participations  in  reform  activities  for  medium-poverty  districts  for  each  of  the  four  sizes/types 
of  districts;  and  the  line  with  data  points  designated  by  triangles  indicates  the  percent  of  participations  in  reform  activities  in  low  poverty  districts 
for  each  of  the  four  sizes/types  of  districts. 


Illustrations  of  Coordination  in  Case-Study  Districts 

Although  our  survey  data  provide  information  about  the  existence  of  coordination  and  co- 
funding, they  do  not  allow  us  to  judge  the  frequency  or  quality  of  the  relationships  between 
Eisenhower  coordinators  and  other  district  administrators.  Case  data,  however,  provide  information 
on  the  nature  and  depth  of  coordination  and  co-funding  between  Eisenhower  coordinators  and 
coordinators  of  other  federally  funded  programs. 

Our  case  districts  provide  examples  of  the  close  coordination  that  can  occur  between 
Eisenhower-assisted  activities  and  those  funded  by  NSF  initiatives.  In  several  of  our  case  districts, 
Urban  Systemic  Initiative  grants,  funded  by  NSF,  greatly  expand  the  funding  available  for 
professional  development  in  mathematics  and  science,  according  to  district  administrators.  Middle 
City,  Wisconsin,  for  example,  has  a particularly  varied  array  of  professional  development  activities 
in  mathematics  and  science,  funded  from  a variety  of  sources.  Thus,  the  district  uses  all  of  its  funding 
sources  together  to  support  the  large  variety  of  professional  development  activities.  These  activities 
rely  heavily  on  mathematics  and  science  resource  teachers,  who  serve  as  mentors  for  the  other 
teachers  in  their  schools.  Professional  development  funds  also  support  semester-long  workshops  for 
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teachers,  opportunities  for  teachers  to  immerse  themselves  in  mathematics  or  science-related  industry 
over  the  summer,  mini-grants  to  teachers  to  follow  up  on  what  they  learned  in  district  workshops, 
and  school-based  staff  development  activities.  While  Eisenhower  funds  contributed  about  $900,000 
in  1996-97  to  such  activities,  the  district’s  USI  contributed  close  to  $3  million  in  that  year  and  was 
considered  an  umbrella  for  all  professional  development  in  mathematics  and  science  in  the  district. 

In  Middle  City,  the  relationships  between  the  USI  and  Eisenhower-assisted  activities  is  more 
than  just  financial,  because  a number  of  the  staff  of  the  USI  have  been  associated  with  Eisenhower- 
assisted  activities  for  years  prior  to  the  district  winning  the  USI.  The  USI’s  leadership  includes  a 
former  Eisenhower  coordinator  and  an  Eisenhower-assisted  mentor  teacher.  In  addition,  many  of  the 
USI’s  mentor  teachers  have  formerly  been  mentor  teachers  funded  by  the  Eisenhower  program.  It 
appears  that,  at  least  in  this  district,  the  Eisenhower  funds  have  over  the  years  developed  leadership 
and  capacity  in  mathematics  and  science  that  provided  a foundation  for  the  USI. 

In  contrast  to  Middle  City,  the  relationship  between  Eisenhower-assisted  activities  and  those 
of  NSF-funded  programs  is  not  close  in  some  other  districts.  Through  the  state’s  SSI,  Richmond, 

New  York’s  math  and  science  teachers  have  access  to  extensive  professional  development 
opportunities,  in  particular  summer  institutes  and  ongoing  teacher  study  groups.  At  the  district  level, 
however,  coordination  between  the  Eisenhower  program  and  the  SSI  is  virtually  nonexistent.  The 
district  distributes  the  bulk  of  its  Eisenhower  funds  directly  to  schools  where  school  staff  determine 
how  they  should  be  used.  In  the  views  of  district  administrators  and  teachers,  professional 
development  related  to  the  SSI  is  separate  from  the  Eisenhower  funds  that  schools  receive,  and 
neither  administrators  nor  teachers  seem  to  see  the  district’s  Eisenhower  funds  as  an  opportunity  to 
build  on  or  expand  opportunities  available  through  the  SSI. 

In  another  district,  South  City,  Florida,  coordination  between  Eisenhower  and  USI  activities 
damaged  professional  development  activities  in  the  short  term,  at  least  from  the  perspective  of  the 
Eisenhower  coordinator  and  some  teachers.  According  to  the  first  report  from  this  evaluation,  South 
City,  Eisenhower  program  activities  were  “subsumed  by”  the  USI  (Birman,  Reeve,  & Sattler,  1998, 
p.  31).  Activities  supported  by  Eisenhower  funds  tended  to  be  shorter  in  duration  and  involve  less 
follow-up  than  they  had  prior  to  their  collaboration  with  the  USI.  The  Eisenhower  coordinator 
attributes  this  to  the  fact  that  the  USI  director  and  staff  are  relatively  inexperienced  in  planning, 
organizing,  and  providing  professional  development.  While  the  USI  director  recognizes  the 
shortcomings  of  the  new  approach,  and  the  Eisenhower  coordinator  looks  forward  to  resolving 
differences  in  their  approaches,  the  working  relationship  between  the  two  was  at  least  initially 
problematic. 

Our  survey  data  clearly  indicate  that  coordination  and  co-funding  are  common  between 
Eisenhower-assisted  and  NSF-funded  activities,  as  we  would  expect  since  both  programs  focus  on 
mathematics  and  science  initiatives.  The  case  of  Middle  City  illustrates  such  collaboration  and  co- 
funding at  its  best.  There,  the  USI  and  Eisenhower  staff  work  collaboratively,  and  the  USI  benefits 
from  the  expertise  and  capacity  built  up  through  the  years  of  Eisenhower  funding.  However,  the 
other  case  study  examples  indicate  that,  while  NSF  funds  can  provide  excellent  opportunities  for 
collaboration  and  pooling  of  resources  with  Eisenhower-assisted  activities,  districts  do  not  always 
effectively  capitalize  on  these  opportunities. 

Consistent  with  our  survey  data,  we  find  fewer  examples  in  our  case  studies  of  coordination 
with  professional  development  funded  by  ED  programs.  One  example  of  good  coordination  with  an 
ED  program  is  in  Weller,  Kentucky.  There,  Eisenhower  and  Goals  2000  funds  are  combined  for  an 
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annual  professional  development  project.  The  project  always  involves  a teacher  demonstration  on  a 
particular  topic  in  mathematics  or  science.  In  1997-98  the  topic  was  finance,  with  primary  grade 
teachers  concentrating  on  money  units  and  related  mathematics  skills  and  secondary  school  teachers 
looking  at  designing  and  marketing  products.  Outside  providers  train  master  teachers;  in  1997-98  the 
training  was  provided  by  the  Kentucky  Math  Council.  Master  teachers  then  develop  two-week 
demonstration  units.  Other  teachers  come  into  a master  teacher’s  classroom  in  the  morning  to  help 
plan,  observe  the  lessons,  and  then  take  the  unit  back  to  their  own  schools  to  experiment  with  it  and 
to  demonstrate  it  for  their  colleagues.  A total  of  approximately  40  teachers,  selected  by  their 
principals,  participated  in  this  activity.  Goals  2000  provided  the  majority  of  funds  for  the  activity, 
and  Eisenhower  funds  helped  pay  for  the  training  of  the  master  teachers. 

Rhinestone,  Texas,  also  has  strong  coordination  in  several  areas.  The  Eisenhower 
coordinator  and  the  Title  I coordinator  in  Rhinestone  are  both  subject-area  specialists,  the  former  in 
math  and  science  and  the  latter  in  language  arts;  two  other  subject-area  specialists  also  work  in  the 
district.  The  four  individuals  discuss  needs  of  specific  schools  and,  on  occasion,  ways  of 
encouraging  curriculum  integration.  Each  week  they  meet  to  discuss  the  use  of  Eisenhower  funds  in 
supporting  Title  I goals,  and  the  Title  I coordinator  co-chairs  the  committee  that  decides  how 
Eisenhower  funds  will  be  used  each  year. 

In  some  other  case  districts,  there  is  little  or  no  coordination  between  district-level 
Eisenhower  and  Title  I staff.  More  than  one  Eisenhower  coordinator  was  surprised  that  we  asked  to 
interview  Title  I coordinators,  telling  us  that  Title  I staff  knew  nothing  about  the  Eisenhower 
program.  One  reason  for  this  is  the  focus  of  Title  I activities.  In  a number  of  districts,  Title  I focused 
exclusively  on  reading  and  language  arts.  In  one  district  where  Title  I focuses  on  mathematics,  its 
focus  is  elementary  mathematics,  while  Eisenhower  funds  are  viewed  as  mainly  for  secondary  school 
teachers. 

In  several  case-study  districts,  the  provision  of  Title  I funds  directly  to  schools  while 
maintaining  Eisenhower  funds  at  the  district  level  is  another  reason  for  limited  coordination  between 
the  two  funding  streams.  In  Lone  Star,  Texas,  the  1997-98  school  year  was  the  first  year  under  a new 
organizational  system,  and  was  a challenging  one  for  teachers  and  administrators.  Because  district 
administrators  were  reorganized  into  subdistricts  and  most  people  had  vastly  altered  responsibilities, 
little  professional  development,  Eisenhower-assisted  or  otherwise,  took  place.  However,  other 
factors  still  point  to  a lack  of  coordination  between  the  Eisenhower  program  and  Title  I.  In  Lone 
Star,  all  Title  I funds  are  used  in  school  wide  projects,  with  funds  devolving  directly  to  the  school 
level.  Eisenhower  funds,  however,  are  maintained  at  the  district  level.  Therefore,  decisions  about 
the  two  programs  are  not  made  by  the  same  people,  or  even  by  people  in  the  same  location.  This  is 
the  case  in  other  districts  as  well,  where  there  are  school-wide  Title  I projects  but  district  control  over 
Title  II  funds. 

Summary:  Alignment  and  Coordination 

Our  results  indicate  that  most  teachers  are  in  districts  where  Eisenhower  coordinators  report 
meeting  their  legislative  responsibilities  to  align  Eisenhower  professional  development  with  state  and 
district  standards  and  assessments,  and  to  coordinate  with  others  in  funding  and  operating 
Eisenhower  professional  development  activities.  Aligning  professional  development  with  standards 
and  assessments  could  contribute  to  a coherent  professional  development  program  that  is  focused  on 
goals  and  objectives  that  are  important  for  student  learning,  and  that  support  high  standards  for 
learning  (Smith  & O’Day,  1991;  Cohen  & Hill,  1998).  There  is,  however,  more  alignment  with 
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standards  than  with  assessments.  This  may  reflect  the  fact  that  the  ESEA  requirement  of  aligned 
assessments  had  not  yet  gone  into  effect  at  the  time  of  our  data  collection,  and  in  some  places, 
assessments  may  not  have  been  well-aligned  to  the  standards;  also,  assessments  have  proven  more 
difficult  to  develop  than  standards.  Further,  state  standards  and  assessments  have  as  strong  a 
relationship  to  Eisenhower-assisted  activities  as  do  the  more  immediate  district  standards  and 
assessments.  One  possible  explanation  for  this  is  that  district  policy  tools  tend  to  reflect  state  policy 
tools,  and  so  they  are  seen  as  interchangeable.  It  may  also  reflect  greater  stakes  associated  with  state 
standards  and  assessments. 

The  high  level  of  co-funding  and  collaboration  with  other  Federal  programs  reported  by  our 
sample  of  districts  suggests  that  Eisenhower  coordinators  are  attempting  to  integrate  the  needs  and 
requirements  of  their  federal  program  in  the  design  of  professional  development  activities.  This 
increases  the  district’s  ability  to  use  funds  efficiently  and  effectively.  Clearly  here,  however,  the 
mathematics  and  science  focus  of  Eisenhower  is  a major  explanatory  factor.  Coordination  and  co- 
funding is  nearly  twice  as  prevalent  with  NSF  mathematics  and  science  initiatives  than  with  ED 
initiatives  that  do  not  share  the  mathematics/science  focus.  Furthermore,  our  case-study  data  indicate 
that  some  of  the  coordination  between  Eisenhower  coordinators  and  those  of  other  ED  programs  may 
be  fairly  superficial.  Eisenhower  coordinators  report  that  coordination  often  consists  of  occasional 
conversations  with  other  program  administrators.  Further,  these  conversations  do  not  necessarily 
include  discussion  of  strategies  for  making  multiple-funding  streams  support  each  other  or 
integrating  professional  development  across  programs. 

Districts  with  a higher  proportion  of  students  in  poverty  are  somewhat  more  likely  to  co-fund 
with  other  federal  programs  and  others  in  professional  development  related  roles.  This  might  be  due 
to  the  fact  that  high-poverty  districts  tend  to  receive  money  from  multiple  federal  programs  whose 
formulas  are  often  designed  to  favor  districts  with  larger  numbers  of  children  from  low-income 
families.  The  existence  of  more  sources  of  funding  creates  the  need  for  more  collaborative  efforts  in 
employing  program  funds  to  aid  in  the  education  of  students  with  special  needs.  Thus,  having 
multiple  sources  of  funding  and  a common  purpose — addressing  the  needs  of  students  in  poverty — 
may  facilitate  coordination  in  higher-poverty  districts. 

A more  pronounced  finding  relates  to  district  size.  Large  districts  are  more  likely  than  other 
districts  to  report  aligning  their  professional  development  with  standards  and  assessments,  and  both 
large  districts  and  consortia  co-fund  with  other  programs  more  than  smaller  districts  do.  Large 
districts  and  consortia  may  be  able  to  incorporate  effective  collaborative  practices  because  of 
economies  of  scale  (e.g.,  investment  in  collaborative  efforts  is  less  expensive  as  the  number  of 
participants  and  programs  increases),  and  also  because  they  have  larger,  better  infrastructures  for 
planning  and  delivering  professional  development.  Also,  large  districts  are  more  likely  to  have 
subject-area  specialists  that  are  integrated  into  the  mathematics/science  community.  By  contrast, 
smaller  districts  may  have  fewer  programs  and  personnel  with  which  to  collaborate,  partly  because  in 
smaller  districts  the  Eisenhower  coordinator  tends  to  assume  several  roles.  Smaller  districts  also  may 
not  have  the  capacity  or  incentive  to  work  across  programs  in  the  way  that  often  becomes  necessary 
in  larger  districts.  If  this  is  the  case,  perhaps  federal  technical  assistance  providers  should  consider 
giving  smaller  districts  more  guidance  to  help  them  organize  and  combine  their  funding  streams,  and 
collaborate  with  other  professional  development  providers  in  their  district. 

The  fact  that  most  of  the  districts  in  our  national  profile  of  Eisenhower  coordinators  report 
engaging  in  alignment,  collaboration,  and  co-funding  indicates  that  they  are  attempting  to  meet  the 
requirements  of  the  legislation  in  these  areas,  and  also  are  engaging  in  practices  that  have  been  linked 
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to  providing  high-quality  professional  development.  The  coordination  and  alignment,  however, 
appears  to  be  greatly  enhanced  when  there  is  a shared  subject-matter  focus,  and  in  larger  districts  and 
districts  with  more  students  in  poverty. 

Implementing  the  Vision:  District  Procedures  for  the 
Continuous  Improvement  of  Professional  Development 

Activities 


Section  Findings 

♦ Less  than  one-third  of  the  nation’s  teachers  are  in  school  districts  that  have  developed 
performance  indicators  to  help  them  track  the  quality  and  effectiveness  of  professional 
development  activities.  Of  those  teachers  who  are  in  districts  that  do  have  indicators, 
less  than  a quarter  are  in  districts  that  collect  data  on  the  indicators. 

♦ The  majority  of  the  nation ’s  teachers  are  in  districts  that  assess  teachers  ’ needs  formally; 
nearly  all  teachers  are  in  districts  that  use  teacher  surveys  to  do  so,  and  most  also  use 
less  formal  methods  such  as  meetings  or  conversations. 

♦ Most  of  the  nation’s  teachers  are  in  districts  that  evaluate  the  effectiveness  of 
Eisenhower-assisted  professional  development  activities.  These  evaluations  almost 
always  involve  a teacher  satisfaction  survey;  they  are  less  likely  to  involve  formally 
observing  teaching  practice  or  assessing  effects  on  student  achievement. 

♦ Districts  provide  more  types  of  guidance  (e.g.,  assistance  in  interpreting  Title  II  rules  and 
in  developing  professional  development  plans)  to  schools  than  to  other  providers  of 
professional  development.  Among  all  types  of  support  that  districts  provide,  they  are  least 
likely  to  focus  on  data-driven  support,  such  as  evaluations  and  indicators. 

The  Eisenhower  legislation  reflects  the  “continuous  improvement”  paradigm  adopted  by  the 
federal  government  for  all  of  its  programs.  Specifically,  the  legislation  states  that  “a  local 
educational  agency  shall  set  specific  performance  indicators  for  improving  teaching  and  learning 
through  professional  development”  (Section  2208(a)(2)).  This  provision  applies  to  all  of  the 
district’s  professional  development,  not  just  the  activities  supported  with  Eisenhower  funds.  The 
legislation  also  requires  that  each  LEA  “submit  a report  to  the  State  every  three  years...  regarding  the 
progress  of  such  agency  toward  performance  indicators...  as  well  as  on  the  effectiveness  of  [the 
LEA’s]  activities. . .”  supported  with  Eisenhower  funds  (Section  2401(b)).  These  requirements  are 
consistent  with  experts’  views  that  accountability  for  the  outcomes  of  professional  development  is  a 
key  component  of  high-quality  professional  development  (Loucks-Horsley  et  al.,  1998). 

Performance  indicators  are  designed  to  evaluate  district  professional  development  efforts  by 
establishing  measurable  benchmarks  to  track  progress  toward  the  district’s  goals  and  objectives.  Of 
course,  having  such  indicators  then  implies  a process  of  decisionmaking  in  school  districts  that  is 
grounded  in  objective  data.  Advocates  of  education  reform  are  increasingly  promoting  the  value  of 
school  districts’  using  data  to  make  decisions  about  directions  for  teaching  and  learning  (e.g., 
Bernhardt,  1998).  Two  provisions  of  the  Eisenhower  legislation  that  have  the  potential  for 
encouraging  districts  to  produce  data  regarding  their  professional  development  activities  are  the 
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provision  for  assessing  the  needs  of  teachers,  and  the  provision  for  assessing  Eisenhower-assisted 
activities. 

The  Eisenhower  legislation  requires  that  the  LEA  include  in  its  application  an  assessment  of 
local  needs  for  professional  development,  as  identified  by  local  education  agency  and  school  staff 
(Section  2208(b)).  This  type  of  needs  assessment  of  teachers  is  an  important  first  step  in  planning  a 
professional  development  strategy  that  accurately  addresses  the  strengths  and  weaknesses  of  teachers 
in  a particular  district  (Loucks-Horsley  et  al.,  1998). 

Evaluating  the  effectiveness  of  Eisenhower-assisted  professional  development  activities  also 
should  contribute  to  districts’  tracking  of  progress  toward  professional  development  goals.  As 
mentioned  above,  the  legislation  requires  districts  to  both  assess  progress  toward  established 
indicators  for  the  district  as  a whole,  as  well  as  to  report  on  the  effectiveness  of  Eisenhower-assisted 
district  activities.  Guskey  (1997)  notes  that  commonly  used  measures  for  evaluating  the  effects  of 
professional  development  include  participants’  reactions  to  the  experience;  participants’  actual  use  of 
knowledge  and  skills  they  have  gained;  and  the  impact  of  participants’  changes  in  knowledge  and 
skills  on  student  learning.  Guskey  argues  that  studies  of  professional  development  ought  to  focus  less 
on  teacher  perceptions  and  reactions  and  place  greater  emphasis  on  teacher  and  student  outcomes  of 
professional  development. 

Thus,  Eisenhower  requirements  for  establishing  indicators,  assessing  the  needs  of  teachers, 
and  evaluating  Eisenhower-assisted  activities  are  some  of  the  ingredients  in  the  legislation  that  could 
support  a data-based  continuous  improvement  process.  Knowledge  and  use  of  indicators  should 
provide  target  goals  and  benchmarks  for  measuring  progress.  Information  about  teachers’  needs 
should  assist  in  setting  the  goals  and  objectives  for  professional  development.  Evaluation  data  should 
provide  one  means  for  determining  whether  professional  development  activities  are  moving  teachers 
toward  these  goals. 

However,  to  have  an  effect  on  the  design  of  professional  development  activities,  continuous 
improvement  means  more  than  establishing  goals,  measuring  progress  toward  these  goals,  and 
evaluating  professional  development  activities.  Continuous  improvement  also  means  communicating 
with  schools  and  teachers  about  state  and  district  standards  and  assessments,  sharing  the  data 
collected  from  needs  assessments  and  evaluation,  and  establishing  goals  and  indicators  for 
professional  development.  The  term  continuous  improvement  implies  a “feedback  loop”  in  which 
data  about  progress  are  part  of  continuous  communication,  and  where  data  become  part  of  a 
discussion  about  strengths  and  weaknesses,  and  future  strategies  and  decisions. 

Thus,  in  addition  to  establishing  indicators,  assessing  teacher’s  needs,  and  evaluating  the 
effectiveness  of  professional  development  activities,  another  important  role  that  districts  may  have  is 
to  offer  guidance,  support,  or  technical  assistance  to  those  who  are  involved  in  planning  and 
implementing  professional  development.  Schools  often  have  a role  in  planning  and  implementing 
professional  development  activities,  and  districts  may  play  a role  in  helping  to  familiarize  schools 
with  the  standards  and  expectations  of  districts,  state  administrators,  and  policy  makers,  as  well  as 
with  data  from  district  data  collection  efforts.  Furthermore,  in  addition  to  school  staff,  district  staff, 
outside  consultants,  or  individuals  from  other  organizations  such  as  teacher  centers  or  regional 
service  centers  operated  in  some  states  may  be  providers  of  professional  development  activities. 

Some  providers  may  be  unfamiliar  with  the  district’s  vision  and  goals.  In  these  cases,  district  data 
about  their  performance,  and  guidance  about  district  goals  and  policies,  could  help  these  providers 
integrate  their  activities  with  district  goals  and  standards.  District  support  to  school  staff  and  other 
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providers  of  professional  development  can  help  to  enable  teachers  and  schools  to  have  professional 
development  choices  that  fit  within  the  state  and  districts’  overall  vision  and  focus.  In  requiring 
districts  to  establish  goals  and  objectives  for  professional  development,  assess  needs  of  teachers, 
track  progress  toward  goals,  and  evaluate  the  effects  of  professional  development  activities,  the 
Eisenhower  legislation  highlights  the  role  of  the  district  in  supporting  professional  development 
activities.  In  these  provisions,  the  legislation  is  consistent  with  recent  literature  on  school  reform, 
which  also  emphasizes  the  critical  role  of  school  districts  in  guiding  professional  development 
(Elmore  & Burney,  1996;  Spillane  & Thompson,  1997). 

In  this  section,  we  use  our  national  profile  of  Eisenhower  coordinators  to  examine  the 
aspects  of  Eisenhower-assisted  professional  development  that  relate  to  performance  indicators,  needs 
assessments,  evaluations  of  Eisenhower-assisted  activities,  and  the  provision  of  guidance  to  schools 
and  professional  development  providers.  We  describe  how  successful  districts  are  in  meeting  the 
Title  II  requirements  for  having  performance  indicators  in  place  and  collecting  information  on  these 
indicators,  and  examine  whether  districts  know  about  and  use  state-level  performance  indicators  in 
their  evaluation  and  improvement  process.  We  describe  the  ways  in  which  districts  collect 
information  about  needs  for  professional  development,  and  whether  and  how  districts  evaluate  their 
Eisenhower-assisted  professional  development  activities.  Finally,  we  describe  the  ways  in  which 
districts  provide  guidance  to  schools  and  professional  development  providers  regarding  Eisenhower- 
assisted  professional  development  activities. 

Presence  and  Use  of  Performance  Indicators 

We  asked  Eisenhower  coordinators  if  their  district  had  developed,  or  was  currently 
developing,  performance  indicators  for  professional  development.  Data  from  the  national  survey  of 
district  Eisenhower  coordinators  are  somewhat  discouraging  with  respect  to  district  response  to  the 
requirements  for  having  and  using  performance  indicators.  Some  district  Eisenhower  coordinators 
continue  to  be  unaware  of  the  requirements,  or,  if  they  are  aware  of  them,  they  have  not  yet  acted  on 
them.  As  Exhibit  5.4  illustrates,  less  than  a third  (32  percent)  of  the  nation’s  teachers  are  in  districts 
that  have  performance  indicators  in  place. 

Of  teachers  who  are  in  districts  that  have  already  developed  performance  indicators  (32 
percent),  60  percent  are  in  districts  that  report  that  they  collect  data  on  those  indicators  to  measure 
progress;  an  additional  40  percent  of  teachers  are  in  districts  that  say  they  have  plans  to  do  so  (data 
not  shown).  In  other  words,  fewer  than  one  in  five,  or  about  1 8 percent,  of  teachers  are  in  districts 
that  currently  collect  data  on  performance  indicators  that  they  have  established  to  guide  their 
professional  development  efforts. 

While  most  of  the  nation’s  teachers  are  in  districts  that  have  not  yet  developed  district-level 
performance  indicators,  district  efforts  to  plan,  evaluate,  and  track  the  progress  of  their  professional 
development  activities  could  benefit,  in  principle,  from  indicators  developed  by  their  states. 
However,  a majority  of  teachers  are  in  districts  where  Eisenhower  coordinators  are  not  aware  of  any 
performance  indicators  developed  at  the  state  level;  only  34  percent  of  teachers  are  in  districts  where 
coordinators  report  that  they  knew  of  state-level  indicators.  Of  these,  70  percent  of  teachers  are  in 
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EXHIBIT  5.4 


Percent  of  Teachers  in  Districts  According  to  Status  of  District  Performance 
Indicators  for  Professional  Development  (n=363) 


indicators  indicators  indicators 

Status  of  district  performance  indicators 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1 998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  32  percent  of  teachers  are  in  districts  that  report  that  they  have  developed 
performance  indicators.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 


districts  in  which  district  Eisenhower  coordinators  report  that  the  state  indicators  affect  their  use  of 
Eisenhower  funds  or  other  district  activities  (data  not  shown).  In  other  words,  less  than  25  percent  of 
the  nation’s  teachers  are  in  districts  that  report  being  affected  by  their  states’  indicators  for 
professional  development. 

Of  the  district  Eisenhower  coordinators  who  say  that  the  state  indicators  affect  their  districts, 
coordinators  report  that  state  indicators  affect  the  district  in  the  following  ways  (these  answers  were 
in  response  to  an  open-ended  question  that  asked  in  what  ways,  if  any,  are  indicators  used  to  guide 
change  in  Eisenhower-assisted  professional  development  activities): 

♦ help  the  district  assess  school,  teacher,  and/or  overall  district  needs, 

♦ help  the  district  focus  and  plan  its  use  of  Eisenhower  funds, 

♦ influence  the  types  and  characteristics  of  Eisenhower-assisted  activities  (i.e.,  duration,  in- 
district versus  out-of-district,  workshop  versus  institute), 

♦ encourage  districts  to  strive  to  meet  state  standards,  and 

♦ provide  a measure  for  districts  to  use  to  evaluate  their  performance. 

In  general,  the  district  case  studies  mirror  results  from  the  district  survey  findings.  There  is  a 
lack  of  awareness  of  the  states’  performance  indicators.  Three  of  the  five  case-study  states  collect 
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data  from  districts  regarding  the  state’s  indicators.  Case-study  districts  in  these  states  supply  the 
requested  information  to  state  coordinators,  but  they  do  not  perceive  the  data  as  part  of  a system  of 
evaluating  progress  toward  state  or  district  goals  for  professional  development.  In  at  least  two  of  the 
case-study  states,  districts  are  required  to  select  a subset  of  the  state  indicators  and  report  their 
progress  on  them.  However,  district  Eisenhower  coordinators  do  not  consider  those  indicators  they 
report  on  to  the  state  to  be  their  district’s  performance  indicators,  and  these  indicators  do  not  appear 
to  be  in  any  way  guiding  the  districts  in  their  planning  and  evaluation  of  professional  development. 
Only  one  of  the  six  districts  in  these  states  had  “developed”  indicators — and  that  district  adopted  a 
subset  of  the  state’s  indicators  as  its  own. 

Assessing  the  Needs  of  Teachers 

In  addition  to  establishing  indicators,  obtaining  information  about  the  needs  of  teachers  is  a 
first  step  in  planning  for  professional  development,  and  as  mentioned  earlier,  the  legislation  requires 
that  districts  conduct  needs  assessments.  We  asked  district  Eisenhower  coordinators  if  teachers’ 
needs  for  professional  development  are  assessed,  and  if  so,  in  which  ways  they  are  assessed  (note: 
this  question  applies  to  all  types  of  professional  development,  not  just  Eisenhower-assisted 
professional  development).  Options  on  the  survey  were:  1)  with  a survey  of  teachers,  2)  with 
meetings  of  teacher  representatives,  3)  with  a survey  of  principals  or  department  chairs,  4)  with 
measures  of  student  performance,  and  5)  with  informal  conversations. 

Just  under  85  percent  of  teachers  are  in  districts  where  coordinators  say  that  they  formally 
assess  teachers’  professional  development  needs  (data  not  shown).  As  Exhibit  5.5  shows,  most  of  the 
districts  use  several  methods  for  assessing  needs.  Teacher  surveys  are  the  most  popular  method;  80 
percent  of  teachers  are  in  districts  that  formally  assess  needs  using  this  method.  Between  70  and  76 
percent  of  teachers  are  in  districts  that  use  meetings  with  teacher  representatives,  surveys  of  the 
principal  or  the  department  chairperson,  and  informal  conversations,  and  65  percent  of  teachers  are  in 
districts  that  use  measures  of  student  performance  to  determine  teachers’  professional  development 
needs.  Our  survey  data  do  not  yield  information  on  how  often  needs  assessments  of  the  various  types 
are  used  or  how  heavily  districts  rely  on  these  different  methods. 

In  our  case-study  districts,  however,  we  find  examples  of  how  these  approaches  to  assessing 
needs  are  used.  Six  of  our  case  sites  conduct  some  type  of  teacher  survey  to  identify  teacher  needs 
for  professional  development.  For  example,  needs  assessment  appears  to  rely  exclusively  on  a 
teacher  survey  in  Weller,  Kentucky.  There,  teachers  are  asked  about  future  needs  for  professional 
development  at  the  end  of  their  evaluation  of  professional  development  activities.  Last  year,  the 
following  were  among  the  most  frequently  mentioned  needs:  more  planning  time  in  order  to 
implement  new  ideas;  information  on  how  to  use  and  integrate  technology  into  the  curriculum; 
strategies  to  work  with  students  with  special  needs;  information  on  working  in  classrooms  with 
students  of  varying  ages  and  ability  levels;  help  in  collaborative  and  cooperative  learning;  time 
management;  new  and  current  literature  on  teaching  and  learning;  and  stress  management.  Such 
information,  and  evaluations  of  teacher  satisfaction  with  professional  development  activities,  are  the 
basis  for  planning  Eisenhower-assisted  activities,  which  in  this  district  is  done  almost  entirely  by  the 
Eisenhower  coordinator  herself. 


EXHIBIT  5.5 


Percent  of  Teachers  in  Districts  That  Use  Different  Methods  to  Assess  Teachers’ 
Professional  Development  Needs  (n=363) 


Needs  assessment  methods 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  80  percent  of  teachers  are  in  districts  that  report  using  teacher  surveys  as  a 
method  of  needs  assessment.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 


These  examples  of  the  variation  of  needs  that  teachers  identify  highlight  the  challenge  in 
aligning  teachers’  needs  with  district  standards,  and  addressing  potential  differences  between  the  two 
sources.  Relying  exclusively  on  teachers’  assessments  of  their  own  needs  may  be  problematic, 
because  teachers’  perceptions  of  their  weaknesses  are  influenced  by  their  instructional  philosophy 
and  goals,  which  may  not  be  consistent  with  district  standards  and  goals. 

Student  test  scores  are  the  main  source  of  needs  assessment  data  in  some  districts,  though 
sometimes  they  are  used  in  combination  with  other  sources  of  information.  For  example,  in 
Rhinestone,  Texas,  the  needs  of  teachers  are  determined  by  a combination  of  informal  conversations, 
in  person  and  by  telephone,  “walk  throughs”  by  district  staff  in  order  to  observe  and  talk  with 
teachers,  and  an  analysis  of  TAAS  scores.  Teachers  call  district  staff  to  express  their  needs,  and  their 
opinions  about  the  types  of  professional  development  that  they  would  like.  But,  according  to  district 
administrators,  academic  achievement,  as  measured  by  the  Stanford  9 and  TAAS  scores,  often 
determine  who  is  targeted  for  professional  development  and  the  type  of  professional  development 
that  is  needed.  District  administrators  indicate  that  reliance  on  student  achievement  tests  to  guide 
professional  development  needs  is  more  possible  for  mathematics  than  science  in  some  districts, 
because  students  are  tested  more  often  in  mathematics  than  in  science. 
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Evaluating  Professional  Development  Activities 


Another  mechanism  for  continuous  improvement  is  evaluation.  Most  teachers  (93  percent) 
are  in  districts  in  which  district  Eisenhower  coordinators  report  that  they  evaluate  their  Eisenhower- 
assisted  professional  development  activities  (data  not  shown).  Given  that  assessment  of  these 
activities  against  performance  indicators  is  required  in  the  authorizing  legislation,  it  is  perhaps 
surprising  that  as  many  as  seven  percent  of  teachers  are  in  districts  that  acknowledge  that  they  do  not 
evaluate  Eisenhower-assisted  professional  development. 

Districts  report  that  they  evaluate  Eisenhower-assisted  professional  development  in  a number 
of  ways.  We  asked  district  coordinators  to  indicate  which  of  the  following  ways  they  evaluate  the 
activities:  1)  by  number  of  teachers  participating  in  professional  development,  2)  with  a teacher 
satisfaction  survey,  3)  with  observations  of  teachers,  and  4)  with  student  achievement  scores.  We  did 
not  ask  the  frequency  with  which  districts  use  these  evaluation  methods;  districts  may  use  some  of 
them  only  occasionally. 

Exhibit  5.6  shows  that  85  percent  of  teachers  are  in  districts  that  evaluate  Eisenhower 
professional  development  activities  using  a teacher  satisfaction  survey. 

EXHIBIT  5.6 


Percent  of  Teachers  in  Districts  That  Use  Different  Methods  to  Evaluate 
Eisenhower-assisted  Activities  (n=363) 
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Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  70  percent  of  teachers  are  in  districts  that  report  using  counts  of  teacher  participation  as  a 
method  for  evaluating  their  Eisenhower-assisted  professional  development  activities.  Each  bar  and  the  number  on  top  of  it  represent  the  percent 
of  teachers  in  districts  for  each  category. 


Approximately  three-fourths  of  teachers  are  in  districts  that  evaluate  activities  by  the  number 
of  teachers  who  participate  (70  percent)  and  by  observing  the  classroom  practices  of  teachers  who 
have  participated  (71  percent).  The  least  popular  method  is  using  student  achievement  data  to 
evaluate  the  effects  of  professional  development,  although  60  percent  of  teachers  are  in  districts  that 
report  using  this  method. 


Our  case  districts  appear  to  reflect  our  national  data  in  the  evidence  for  reliance  on  teacher 
satisfaction  surveys  to  evaluate  the  outcomes  of  professional  development  activities.  For  example, 
Maple  City,  Ohio,  uses  a standard  teacher  satisfaction  form  to  evaluate  professional  development 
activities.  The  form  asks  teachers  whether  they  agree  with  the  following  statements: 

1 . This  inservice  dealt  with  a priority  issue. 

2.  The  goal(s)  of  the  inservice  were  clearly  stated  by  the  presenter(s). 

3.  The  stated  goals(s)  of  the  inservice  were  achieved. 

4.  The  information  was  clearly  presented. 

5.  I will  modify  my  teaching  behavior  as  a result  of  this  inservice. 

6.  The  information  presented  will  assist  me  in  my  job. 

7.  A portion  of  the  inservice  time  was  set  aside  for  questions  and  answers. 

8.  I believe  there  was  sufficient  input  from  staff  in  planning  this  inservice. 

Teachers  are  also  asked  to  respond  to  three  open-ended  questions: 


1 . What  was  the  most  valuable  part  of  this  inservice? 

2.  In  what  ways  could  this  inservice  have  been  improved? 

3.  Additional  comments. 


In  other  case  sites,  as  well,  evaluation  of  professional  development  involved  teacher 
satisfaction  surveys  that  asked  questions  similar  to  those  on  the  Maple  City  survey. 

While  our  national  sample  of  district  Eisenhower  coordinators  frequently  cites  the  use  of 
observation  of  teachers  and  student  achievement  data  as  a means  for  evaluation,  our  case-study  data 
indicate  that  districts  do  not  observe  teachers  or  use  student  data  in  a formal,  systematic  way  to 
evaluate  the  outcomes  of  professional  development  activities.  Boonetown,  Kentucky,  is  not  unusual 
in  its  use  of  student  assessments  to  evaluate  professional  development  activities.  There,  in  addition 
to  teacher  satisfaction  surveys,  professional  development  is  considered  successful  if  KIRIS  scores 
increase  in  the  targeted  areas.  While  this  is  an  outcome-based  strategy  for  evaluation,  it  can  be 
considered  a remediation  approach  rather  than  a capacity-building  approach;  it  appears  to  neglect  two 
critical  central  steps — an  objective  assessment  of  the  quality  of  the  professional  development  activity, 
and  the  evaluation  of  the  effects  of  professional  development  on  teaching  itself  In  neglecting  these 
two  areas,  the  attribution  of  test  score  increases  to  Eisenhower  professional  development  is  tenuous. 

In  the  sequence  of  steps  from  needs  assessment,  to  setting  goals,  to  designing  and  participating  in 
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professional  development,  to  changes  in  teaching,  to  student  learning,  to  student  outcomes,  we  find 
that  none  of  our  case  districts  systematically  observe  teachers’  practices  after  they  have  participated 
in  professional  development  activities  in  a systematic  way. 

Some  coordinators  in  our  case  districts  state  that  they  observe  teachers,  but  their  observations 
appear  to  be  informal.  For  example,  in  Rhinestone,  Texas,  the  Eisenhower  coordinator  visits 
classrooms  to  observe  teachers  who  have  recently  been  through  training  to  see  if  new  ideas  are  in  use 
in  the  classroom.  On  the  whole,  her  evaluation  seemed  cursory  to  our  case  visitors.  While  she  visits 
each  classroom  and  uses  an  NCTM  checklist  of  effective  classroom  practices  for  criteria,  she  relies 
primarily  on  anecdotal  notes  to  record  her  observations.  She  explains  that  her  goal  is  not  to  make 
teachers  feel  that  they  are  being  evaluated,  but  rather  to  send  a message  of  assistance  and  support  to 
teachers.  Further,  as  we  reported  earlier,  in  this  and  other  districts,  one  criterion  for  evaluating 
professional  development  in  the  schools  is  students’  academic  achievement.  However,  without 
rigorous,  longitudinal,  well-designed  studies,  it  is  difficult  to  attribute  students’  test  score  increases  to 
teachers’  professional  development  experiences. 

District  Support  and  Guidance  to  Schools  and  Professional  Development 
Providers 

Continuous  improvement  means  setting  goals,  collecting  and  reporting  measurable  indicators 
of  progress,  assessing  the  needs  of  teachers,  and  evaluating  professional  development  activities. 
However,  it  also  means  communicating  goals  and  evaluation  results  with  those  who  provide 
professional  development,  so  that  they  are  able  to  incorporate  district  goals  into  their  professional 
development  plans,  and  know  how  their  activities  will  be  judged.  Professional  development  activities 
often  are  provided  by  schools,  or  by  district  staff  or  individuals  hired  by  the  district.  Institutions  of 
higher  education  and  nonprofit  organizations  also  frequently  provide  professional  development  to 
teachers  in  districts;  and  districts  may  work  with  these  organizations  in  various  ways  to  offer  them 
guidance  and  support. 

To  find  out  about  the  types  of  support  and  guidance  that  districts  offer  to  schools  and 
professional  development  providers,  we  asked  district  Eisenhower  coordinators  a number  of 
questions  about  the  types  of  communication  they  have  with  schools  and  with  providers  of 
professional  development  and  the  guidance  they  provide  to  schools  and  other  professional 
development  providers.  Specifically,  we  asked  coordinators  in  which  of  the  following  ways  district 
Eisenhower  staff  exchange  information  with  schools  regarding  professional  development:  regular 
visits  and  observations,  telephone  calls,  required  reports,  and/or  required  evaluations.  We  also  asked 
them  which  of  the  following  five  types  of  assistance  they  provide  to  schools:  guidance  in  interpreting 
Title  II  rules,  help  conducting  needs  assessments,  help  developing  professional  development  plans, 
help  developing  specific  activities,  and  help  developing  performance  indicators  for  professional 
development. 

On  average,  districts  provide  more  guidance  to  schools  than  to  providers  of  professional 
development.  Districts  report  providing  6.2  of  the  nine  types  of  support  to  schools  and  4.6  to 
providers.  Standard  deviations  are  quite  large  (2.6  for  schools  and  3.4  for  providers),  which  suggests 
that  there  is  a large  variation  in  the  number  of  ways  that  districts  provide  guidance  (data  not  shown). 

As  illustrated  in  Exhibit  5.7,  districts  are  about  equally  likely  to  provide  support  to  schools  as 
to  providers  in  the  form  of  classroom  visits,  phone  calls,  and  required  reports  and  evaluations. 
However,  teachers  are  much  less  likely  to  be  in  districts  that  provide  the  other  types  of  guidance  (i.e., 


interpreting  Title  II  rules,  conducting  needs  assessments,  and  developing  professional  development 
plans  and  activities  indicators)  to  providers  of  professional  development  than  to  schools.  Also, 
districts  provide  fewer  data-related  types  of  support  (e.g.,  reports  and  evaluations)  than  other  types  of 
support,  to  both  schools  and  professional  development  providers.  Although  these  data  provide 
information  about  whether  or  not  district  coordinators  engage  in  particular  types  of  support  and 
guidance,  we  have  no  information  on  the  quality  or  frequency  of  these  activities. 

EXHIBIT  5.7 


Percent  of  Teachers  in  Districts  in  Which  Eisenhower  Staff  Provide  Different  Types  of 
Guidance  about  Professional  Development  to  Schools  and  Professional  Development 

Providers  (n=363) 


assessment 


□ Guidance  to 
Schools 

■ Guidance  to 
Providers 


Types  of  guidance 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  64  percent  of  teachers  are  in  districts  that  report  using  classroom  visits  as  a type  of  guidance 
that  Eisenhower  staff  provide  to  their  schools,  and  53  percent  of  teachers  are  in  districts  that  report  using  classroom  visits  as  a type  of  guidance  to 
professional  development  providers.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 

To  measure  the  extent  to  which  districts  differ  on  all  continuous  improvement  measures,  we 
created  a composite  of  indicators,  needs  assessment,  evaluation,  and  guidance  to  schools  and 
professional  development  providers.  We  standardized  the  variables  comprising  each  of  these  five 
measures,  adjusted  the  values  to  be  on  a zero  to  one  scale,  and  then  summed  them  to  form  an  overall 
continuous  improvement  composite.  Exhibit  5.8  shows  that  there  is  significant  variation  in 
continuous  improvement  according  to  both  district  size  and  poverty  level.  Small  districts  employ 
significantly  fewer  continuous  improvement  efforts  than  either  consortia  or  large  and  medium 
districts,  and  medium  districts  employ  significantly  fewer  continuous  improvement  efforts  than  large 
districts.  Similarly,  low-poverty  districts  have  fewer  continuous  improvement  mechanisms  than 
high-poverty  districts.  Again,  these  results  may  reflect  that  large  districts  have  more  staff  and  a more 
comprehensive  professional  development  program,  which  requires  more  systematic  monitoring  and 
evaluation.  Similarly,  high-poverty  districts  are  more  likely  to  have  support  from  other  federal 
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programs,  such  as  Title  I,  Part  A,  which  also  emphasize  the  use  of  indicators,  needs  assessments  and 
evaluation. 


EXHIBIT  5.8 

Extent  of  District  Continuous  Improvement  Efforts,  Overall  and  by  District  Poverty 

and  District  Size  (n=363) 
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Significant  Pairwise  Contrasts 

Size  Small  vs.  Medium,  Small  vs.  Large;  Small  vs.  Consortium;  Medium  vs.  Large 

Poverty Low  vs.  High 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  teachers  are  in  districts  that  report  an  average  continuous  improvement  score  of 
2.6,  where  0 indicates  no  continuous  improvement  efforts,  and  five  indicates  the  largest  extent  of  continuous  improvement  efforts. 
Continuous  improvement  differs  significantly  by  both  district  poverty  level  and  district  size.  Each  dot  represents  one  district.  As  the 
number  of  districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  increases  in  length.  Each  distribution  represents 
the  distribution  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


Summary:  Continuous  Improvement 

Most  districts  do  not  have  performance  indicators  and  many  remain  unaware  of  the 
requirement  to  develop  performance  indicators  and  of  the  purposes  underlying  state  indicators  and 
data  collection  activities,  although  Congress’  intent  is  clear  in  these  provisions  to  establish  and 
measure  progress  on  performance  indicators.  Strategic  planning  that  includes  developing  goals  and 
objectives  and  delineating  how  progress  toward  achieving  these  goals  will  be  measured  is  key  to  a 
successful  professional  development  program  (Loucks-Horsley  et  al.,  1998).  Clearly,  there  is  room  in 
this  area  for  a heightened  federal  role  in  assisting  states  and  districts  to  learn  how  to  develop  well- 
constructed  indicators,  how  to  collect  meaningful  data,  and  how  to  use  the  indicators  as  part  of  a 
system  of  continuous  improvement. 

Our  national  survey  data  also  show  that  while  most  districts  collect  information  about  the 
needs  of  teachers,  and  evaluate  Eisenhower-assisted  professional  development  activities,  as  required 
by  law,  their  approaches  to  these  activities  are  not  as  strong  as  they  could  be.  Our  survey  and  case- 
study  data  reveal  that  district  needs  assessments  and  evaluations  are  typically  based  on  teacher 
surveys.  This  heavy  reliance  on  teacher  reports  does  not  take  into  account  the  potential  differences 
between  teachers’  instructional  goals  and  district  goals  and  standards.  Further,  teachers  may  not  be 
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able  to  identify  their  own  weaknesses,  and  even  if  they  can,  they  may  not  be  able  to  identify 
appropriate  measures  to  address  the  weaknesses. 

Districts  also  tend  to  use  classroom  observations  and  student  achievement  data  in  needs 
assessment  and  evaluation,  but  not  in  rigorous  and  systematic  ways.  Most  districts  do  not  have  the 
resources  to  use  these  approaches  effectively.  Studies  that  link  professional  development  with 
student  outcomes  would  require  costly  long-term  data  collections  and  an  evaluation  capacity  that 
many  districts  do  not  have;  these  types  of  evaluations  are  better  conducted  as  part  of  well-designed 
research  studies.  It  seems  more  appropriate  and  effective  for  districts  to  focus  their  evaluations  on 
assessing  how  well  their  professional  development  activities  reflect  high-quality  structural  and  core 
features.  This  can  be  done  through  carefully  crafted  teacher  surveys  that  ask  teachers  specific 
questions  about  the  contact  hours,  duration,  active  learning  opportunities,  content  focus,  and 
coherence  of  the  activity,  as  well  as  through  direct  observation  of  the  activity. 

Districts  do  often  provide  guidance  to  schools  and  professional  development  providers. 
However,  despite  the  need  to  use  data  to  make  decisions  about  professional  development  activities, 
districts  do  not  appear  to  communicate  often  to  schools  and  professional  development  providers 
about  indicators  and  evaluations.  There  are  a number  of  possible  reasons  for  this.  First,  district 
Eisenhower  coordinators  themselves  may  not  be  trained  as  evaluators,  and  may  not  be  familiar  with 
uses  of  data  to  judge  their  progress.  Second,  a number  of  Eisenhower  coordinators  in  our  case 
districts  indicate  that  they  are  aware  of  the  deficiencies  of  their  approaches,  but  do  not  have  adequate 
resources  to  conduct  more  thorough  evaluation  and  data  collection  activities.  Considering  the 
critical  nature  of  the  requirements  to  establish  indicators  and  to  evaluate  Eisenhower-assisted 
activities,  it  may  be  helpful  to  determine  more  definitively  what  the  barriers  are  to  establishing 
indicator  systems  and  high-quality  evaluations,  and  to  provide  guidance  and  assistance  to  help  the 
districts  overcome  these  barriers. 

The  Role  of  Teachers  in  Planning  Professional 

Development 


Section  Findings 

♦ Most  teachers  are  in  districts  that  report  professional  development  being  planned  at  both 
the  school  and  district  levels . 

♦ Most  teachers  are  in  districts  where  teachers  are  involved  in  planning  professional 
development  in  a variety  of  ways,  such  as  through  teacher  committees,  needs 
assessments,  or  informal  consultation . 

♦ While  most  districts  rely  on  schools  to  plan  some  professional  development  activities, 
districts  vary  widely  in  their  interpretations  of  the  “80-20”  rule.  Some  districts  interpret 
the  rule  to  require  that  80  percent  of  funds  be  allocated  directly  to  schools.  Others 
interpret  the  rule  to  mean  that  most  Eisenhower  activities  occur  at  the  school  site.  Still 
other  districts  interpret  the  rule  to  allow  80  percent  of  activities  to  be  determined  by 
district-wide  assessments  of  teacher  needs.  Currently,  all  of  these  interpretations  are 
correct,  according  to  the  Department  of  Education. 
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Aligning  professional  development  activities  with  standards  and  assessments,  and 
coordinating  with  activities  funded  by  other  programs,  can  be  part  of  building  a vision  for  how 
professional  development  can  support  school  reform.  Once  such  a vision  is  established,  continuous 
improvement  activities,  such  as  establishing  indicators  and  collecting  data  about  teacher  needs  and 
the  effectiveness  of  professional  development  activities,  can  be  important  to  implementing  this 
vision.  Another  aspect  of  building  and  implementing  a vision  for  professional  development  is 
involving  teachers  in  planning  professional  development  activities. 

The  Eisenhower  legislation  pays  special  attention  to  involving  teachers  and  school  staff  in 
planning  professional  development  activities.  Such  planning  can  occur  at  any  level  of  the  formal 
school  district  hierarchy,  and  can  involve  teachers  and  other  school  staff  in  a variety  of  roles.  For 
example,  administrators  can  plan  professional  development  activities  at  the  district  level,  with  the 
advice  of  teachers,  or  using  information  from  teachers  about  their  needs.  Alternatively,  professional 
development  can  be  planned  at  the  school  level,  by  full  faculties  or  by  teacher  representatives,  or  by 
principals  without  the  involvement  of  teachers.  Additionally,  in  school  districts,  decisions  about 
professional  development  may  be  made  at  some  level  in  between  the  district  and  the  school,  by 
“clusters”  of  schools.  These  clusters  may  be  “feeder  patterns”  (a  high  school  and  the  elementary  and 
middle  schools  that  “feed”  it),  or  they  may  be  some  other  group  of  schools  (e.g.,  all  high  schools  or 
all  elementary  schools  in  the  district).  As  with  district  and  school-level  decisions,  cluster-level 
decisions  can  involve  teachers  in  a variety  of  roles.  At  whatever  level,  the  goal  of  planning  for 
professional  development  is  to  design  activities  and  experiences  that  improve  the  quality  of  teaching 
and  learning  by  supporting  the  needs  of  teachers. 

Whether  the  planning  for  professional  development  occurs  at  the  district,  cluster,  or  school 
levels,  the  odds  of  meeting  teacher  needs  are  increased  if  teachers  are  involved  in  the  planning. 
Experts  agree  that  teacher  involvement  in  planning  contributes  to  high-quality  professional 
development  (Clark,  1992).  Teacher  involvement  in  planning  can  help  ensure  that  professional 
development  addresses  the  skills  that  they  need,  and  employ  the  learning  strategies  that  they  find 
most  useful  (Clark,  1992;  Loucks-Horsley  et  al.,  1998). 

Echoing  this  idea,  the  Eisenhower  legislation  calls  for  involving  teachers  and  other  school- 
level  staff  in  planning  professional  development  activities  at  the  district  level  and  Eisenhower- 
assisted  activities  at  the  school  levels.  The  law  requires  that  each  LEA’s  professional  development 
plan  be 

...developed  with  the  extensive  participation  of  administrators,  staff,  and  pupil  services 

personnel,  which  teachers  shall  also  be  representative  of  the  grade  spans  within  schools  to  be 

served  and  of  schools  which  receive  assistance  under  part  A of  Title  I (Section  2208(c)(2)). 

However,  the  needs  of  teachers  as  individuals  may  not  be  the  only  needs  that  professional 
development  activities  should  address.  Recent  literature  suggests  that  professional  development 
should  support  both  the  needs  of  individual  teachers  and  those  of  schools  as  a whole  (Loucks- 
Horsley,  1998).  Much  of  the  current  research  base  on  school  reform  emphasizes  the  importance  of 
approaches  that  involve  improving  whole  schools  (e.g.,  Bodilly  et  al.,  1996,  1998;  Herman  et  al., 
1999;  Slavin  et  al.,  1996).  Having  all  teachers  in  a school  share  a knowledge  base,  as  well  as  share 
expectations  for  teaching  and  learning,  facilitates  teaching  and  learning  toward  high  standards 
(O’Day  & Smith,  1993).  In  order  to  improve,  schools  may  have  to  address  needs  that  go  beyond  the 
needs  of  their  individual  teachers.  Furthermore,  researchers  and  policy  makers  are  currently 
emphasizing  the  importance  of  professional  development  that  is  embedded  in  the  daily  life  of 
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teachers  (Guskey,  1997;  Loucks-Horsley  et  al.,  1998).  As  a result,  they  have  focused  on  the  school 
as  the  logical  unit  for  both  planning  and  implementing  ongoing  professional  development  activities 
(Guskey,  1997;Senge,  1990;  Shanker,  1990). 

Reflecting  this  view,  the  Eisenhower  legislation  appears  to  stress  the  importance  of  school- 
level  planning  and  implementation  of  Eisenhower-assisted  activities,  in  addition  to  requiring  that 
teachers  participate  in  planning  Eisenhower-assisted  activities.  The  legislation  specifically  states  that 
LEAs 


shall  use  not  less  than  80  percent  of  such  [Eisenhower  Professional  Development  Program] 
funds  for  professional  development  of  teachers,  and,  where  appropriate,  administrators,  and, 
where  appropriate,  pupil  services  personnel,  parents,  and  other  staff  of  individual  schools  in  a 
manner  that  (A)  is  determined  by  such  teachers  and  staff;  [and]  (B)  to  the  extent  practicable, 
takes  place  at  the  individual  school  site  (Section  2210(a)(1)).  [emphasis  added] 

This  provision,  which  we  refer  to  as  the  “80-20  rule,”  seems  to  reflect  the  Congress’ 
conclusion  that  decisions  about  professional  development  are  best  made  by  school-level  staff. 

This  section  of  the  chapter  examines  how  districts  plan  professional  development  activities, 
with  a focus  on  the  roles  of  school  staff,  particularly  teachers.  We  present  findings  on  the  levels  at 
which  planning  for  professional  development  occurs  and  how  districts  involve  teachers  in  such 
activities.  We  then  discuss  how  districts  interpret  the  legislation’s  provisions  that  deal  with 
participation  of  school  staff  in  planning  professional  development. 

Levels  of  Planning  for  Professional  Development 

We  asked  coordinators  to  tell  us  how  many  ( i.e.,  “none,”  “some,”  “most,”  or  “all”)  of  their 
district’s  Eisenhower-assisted  activities  are  planned  at  the  district,  school,  and  cluster  level.  Exhibit 
5.9a  indicates  the  proportion  of  teachers  who  are  in  districts  where  activities  are  planned  at  each  of 
these  three  levels. 

As  Exhibit  5.9a  shows,  43  percent  of  teachers  are  in  districts  where  Eisenhower  coordinators 
estimate  that  “some”  of  their  Eisenhower-assisted  activities  are  planned  at  the  district  level,  and  34 
percent  are  in  districts  where  “most”  are.  Only  seven  percent  of  teachers  are  in  districts  where  no 
Eisenhower-assisted  activities  are  planned  at  the  district  level,  and  16  percent  are  in  districts  where 
all  activities  are  planned  at  the  district  level. 

Over  half  of  all  teachers  (57  percent)  are  in  districts  that  are  not  organized  in  clusters  for 
planning  and  administering  professional  development,  and  thus  report  no  planning  at  the  cluster 
level.  Thirty-six  percent  of  teachers  are  in  districts  that  plan  “some”  professional  development  at  the 
cluster  level,  but  only  five  percent  are  in  districts  that  plan  “most”  activities  at  the  cluster  level,  and 
two  percent  are  in  districts  in  which  “all”  of  the  professional  development  is  planned  at  the  cluster- 
level. 


Exhibit  5.9a  also  shows  that  48  percent  of  teachers  are  in  districts  where  “some”  Eisenhower- 
assisted  activities  are  planned  at  the  school  level,  while  23  percent  of  teachers  are  in  districts  where 
“most”  of  these  activities  are  planned  at  this  level.  Nineteen  percent  of  teachers  are  in  districts  where 
no  professional  development  activities  are  planned  at  the  school  level,  while  10  percent  of  teachers 
are  in  districts  where  “all”  such  activities  are  planned  at  the  school  level. 
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EXHIBIT  5.9a 


Percent  of  Teachers  in  Districts  Reporting  That  None,  Some,  Most,  or  All  Professional 
Development  Activities  Are  Planned  at  the  District,  School,  and  Cluster  Levels 

(n=363) 


Level  of  planning 


□ None 

□ Some 

□ Most 
■ All 


Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  seven  percent  of  teachers  are  in  districts  that  do  not  plan  any  of  their  activities  at  the 
district  level;  43  percent  of  teachers  are  in  districts  that  plan  some  of  their  professional  development  at  the  district  level;  34  percent  of 
teachers  are  in  districts  that  plan  nu>st  of  their  professional  development  at  the  district  level;  and  16  percent  of  teachers  are  in  districts 
that  plan  all  of  their  professional  development  activities  at  the  district  level.  Each  bar  represents  the  average  percent  of  teachers  in 
districts  for  each  category.  The  number  at  the  top  of  each  section  of  the  bar  is  the  mean. 


In  short,  a small  proportion  of  teachers  are  in  districts  that  plan  all  of  their  professional 
development  activities  at  the  school  or  district  level.  Many  districts,  however,  plan  professional 
development  activities  at  both  the  district  and  school  level. 

To  examine  whether  there  are  differences  in  school-level  and  district-level  planning 
according  to  poverty  level  and  size  of  the  district,  we  created  a measure  of  school-level  vs.  district- 
level  planning.  It  is  a scale  from  0 to  4,  where  four  indicates  districts  where  all  planning  is  done  at 
the  school  level  and  no  planning  is  done  at  the  district  level,  and  0 indicates  districts  where  no 
planning  is  done  at  the  school  level  and  all  planning  is  done  at  the  district  level.  Medium  and  small 
districts  are  more  likely  than  consortia  to  plan  activities  at  the  school  level;  but  there  are  no 
differences  by  poverty,  as  illustrated  in  Exhibit  5.9b.  It  makes  sense  that  consortia  would  be  more 
likely  than  single  districts  to  plan  at  the  district  level,  since  consortia  by  definition  are  a group  of 
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districts  working  together.  Further,  since  consortia  may  be  more  focused  on  across-district  planning, 
school-level  planning  may  not  be  emphasized  as  much  as  in  single  districts. 


EXHIBIT  5.9b 

Extent  to  Which  Professional  Development  Activities  Are  Planned  at  the  School  vs. 
District  Level,  Overall  and  by  District  Poverty  and  District  Size  (n=363) 
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Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  are  in  districts  that  report  an  average  school-level 
planning  score  of  2.3,  where  zero=all  district  level  planning  and  four=all  school  level  planning.  The  extent  of  school-  vs.  district-level 
planning  differs  significantly  by  district  size  but  not  by  district  poverty  level.  Each  dot  represents  one  district.  As  the  number  of 
districts  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the 
distribution  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


Teacher  Involvement  in  Planning 

Involving  teachers  in  different  aspects  of  planning  is  important  for  fostering  high-quality 
professional  development,  and  is  emphasized  by  the  Eisenhower  legislation.  To  find  out  how  and  to 
what  extent  teachers  are  included  in  planning,  we  asked  district  Eisenhower  coordinators  if  teachers 
are  included  in  planning  Eisenhower-assisted  professional  development,  and  in  what  ways  they  are 
included  at  each  level  (i.e.,  district,  cluster,  and  school).  We  asked  which  of  the  following  describes 
the  roles  teachers  play  in  making  decisions  about  Eisenhower-assisted  activities  planned  at  the 
district  level:  1)  teachers  participate  in  a formal  planning  committee,  2)  teachers  are  consulted 
informally,  3)  teachers  are  consulted  in  a needs  assessment,  and/or  4)  teachers  do  not  play  a regular 
role. 


Virtually  all  teachers  (99  percent)  are  in  districts  that  report  that  teachers  are  involved  in  the 
planning  process  (data  not  shown).  Of  teachers  in  districts  that  report  any  teacher  involvement  in 
planning,  88  percent  of  teachers  are  in  districts  that  involve  teachers  in  district-level  planning  through 
needs  assessments  and  informal  consultation  (data  not  shown).  A much  smaller  proportion  of 
teachers,  65  percent,  are  in  districts  that  report  having  teachers  directly  involved  in  district-level 
planning  by  participating  on  formal  committees  (data  not  shown).  Planning  through  membership  on 
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a formal  committee  can  be  considered  to  be  a much  more  active  form  of  involvement  in  planning 
than  through  needs  assessment  or  informal  communication;  thus,  the  most  direct  and  formal  type  of 
teacher  involvement  is  the  least  common. 

We  asked  coordinators  which  of  the  following  people  participate  in  making  decisions  about 
Eisenhower-assisted  activities  planned  at  the  school  level:  1)  lead  teachers,  resource  teachers,  or 
department  chairs,  2)  classroom  teachers,  through  a formally  organized  committee,  and  3)  teachers  as 
individuals.  We  asked  the  same  questions  about  cluster- level  planning. 

The  findings,  reported  in  Exhibit  5.10,  suggest  that  most  districts  involve  teachers  in  the 
school-  or  cluster-level  planning  process  in  more  than  one  way,  and  that  teachers  are  more  involved 
in  school-level  planning  than  cluster-level  planning.  For  example,  77  percent  of  teachers  are  in 
districts  in  which  lead  teachers,  resource  teachers,  or  department  chairs  participate  in  school-level 
planning,  while  only  40  percent  of  teachers  are  in  districts  that  report  participation  of  this  type  of 
teacher  in  cluster-level  planning.  For  both  cluster-  and  school-level  planning,  the  most  likely  to 
participate  are  lead  teachers,  resource  teachers,  or  department  chairs,  followed  by  teachers  as 
individuals.  Once  again,  as  with  district-level  planning,  participation  on  a teacher  committee  is  the 
least  common  form  of  teacher  participation;  62  percent  of  teachers  are  in  districts  that  report  teacher 
participation  in  planning  through  formal  committees  for  school-level  planning.  Thirty  percent  of 
teachers  are  in  districts  that  report  planning  participation  through  formal  committees  for  cluster-level 
planning. 

Because  we  asked  about  teachers’  formal  involvement  on  committees  in  planning 
Eisenhower-assisted  professional  development  at  the  district,  cluster,  and  school  levels,  we  are  able 
to  examine  differences  in  this  type  of  involvement  across  levels.  This  formal  involvement  on 
committees  often  represents  more  responsibility  and  potential  influence  for  the  teacher  than  needs 
assessments  or  informal  involvement.  For  all  districts,  we  find  that  65  percent  of  teachers  are  in 
districts  in  which  teachers  participate  in  formal  committees  at  the  district  level  (data  not  shown),  and 
62  percent  of  teachers  are  in  districts  in  which  teachers  participate  in  formal  committees  at  the  school 
level;  this  difference  is  not  statistically  significant  (data  not  shown).  Thus,  teachers  are  just  as  likely 
to  participate  in  formal  committees  at  the  district  level  as  the  school  level. 

Further,  for  the  districts  in  our  sample  with  clusters,  69  percent  of  teachers  are  in  districts  in 
which  teachers  participate  in  formal  committees  at  the  district  level,  56  percent  are  in  districts  in 
which  teachers  participate  at  the  cluster  level,  and  69  percent  are  in  districts  in  which  teachers 
participate  at  the  school  level  (data  not  shown).  Here  the  differences  between  clusters  and  districts, 
and  clusters  and  schools,  are  statistically  significant,  indicating  that  teachers  are  significantly  less 
likely  to  participate  in  committees  at  the  cluster  level  than  at  either  the  school  or  district  level.  These 
findings  reveal  no  evidence  of  greater  teacher  involvement  as  planning  gets  closer  to  the  school  level, 
since  teachers’  formal  involvement  in  planning  is  similar  when  professional  development  is  planned 
either  at  the  school  or  district  level;  but  cluster-level  planning  does  not,  on  average,  involve  as  much 
teacher  participation.  This  may  reflect  the  organizational  or  management  structure  of  cluster-level 
decision-making,  which  may  be  more  likely  to  have  district  staff  and  administrator  involvement 
because  of  the  need  to  address  issues  across  schools  or  groups  of  schools.  However,  district-level 
planning  also  must  address  across-school  issues,  so  it  is  unclear  why  cluster-level  planning  involves 
significantly  less  formal  teacher  participation. 


I V 

JL 


5-38 


EXHIBIT  5.10 


Percent  of  Teachers  in  Districts  with  Different  Types  of  Teacher  Involvement  in 
School-  and  Cluster-level  Professional  Development  Planning  (n=363) 


Lead  teachers/  Teachers  committee  Individual  teachers 
department  chairs 


Types  of  teacher  involvement 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  77  percent  of  teachers  are  in  districts  where  teachers/department  chairs  are  involved  in  school- 
level  planning.  The  second  bar  shows  that  40  percent  of  teachers  are  in  districts  where  teachers/department  chairs  are  involved  in  cluster-level 
planning.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  teachers  in  districts  for  each  category. 

In  order  to  test  whether  formal  teacher  involvement  in  planning  through  committees  differs 
according  to  the  district  poverty  level  or  district  size,  we  developed  a scale  to  measure  the  overall 
extent  of  this  form  of  teacher  involvement  in  planning.  The  measure  is  an  additive  composite  of 
district  reports  of  teachers’  participation  through  formal  committees  at  the  district,  school,  and  cluster 
levels.  The  composite  is  a function  of  whether  teachers  are  involved  in  planning  at  each  of  the  three 
levels,  weighted  by  the  extent  to  which  activities  are  planned  at  the  three  levels  (i.e.,  coordinators 
indicated  whether  some,  most,  or  all  of  their  activities  were  planned  at  a particular  level).  Analysis 
of  the  composite  indicates  that  on  average,  teachers  are  in  districts  that  involve  teachers  in  the 
planning  process  through  committees  in  two-thirds  of  the  levels  at  which  activities  are  planned  (data 
not  shown).  There  are  no  statistically  significant  differences  according  to  district  poverty  level  and 
district  size. 

Our  case  data  illustrate  how  different  districts  involve  teachers  in  planning  professional 
development  activities.  A few  of  our  case  districts  rely  only  on  teacher  needs  assessment  surveys  for 
planning  professional  development  activities,  sometimes  augmented  by  informal  conversations  with 
teachers.  In  other  districts,  teachers  take  a more  active  role  as  members  of  teacher  committees.  Data 
from  the  following  case  districts  illustrate  how  teachers’  committees  can  be  involved  in  planning  for 
professional  development  at  the  school  or  district  levels. 

In  West  City,  California,  a district  committee,  or  “implementation  team,”  of  up  to  100 
individuals  meets  to  plan  a different  reform  effort  every  year,  including  professional  development 
activities.  After  the  district  has  identified  one  area  of  greatest  need  a year,  based  on  student  and 
school  data,  current  state  or  district  reform  efforts,  or  plans  for  textbook  adoption,  the 
implementation  team  meets  to  plan  the  reform  effort,  including  professional  development.  The 
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implementation  team  includes  district  staff,  teachers,  IHE  staff,  representatives  of  community 
organizations,  and  experts  in  the  field.  Once  the  implementation  team  develops  a draft  plan,  the  team 
seeks  input  from  various  constituent  groups,  and  holds  focus  groups  to  fine-tune  the  plan.  With  these 
inputs,  the  draft  plan  is  completed  and  disseminated  district-wide.  The  plan,  including  its 
professional  development  component,  continues  to  be  a working  document,  and  is  modified  if 
necessary  as  the  reform  proceeds. 

Northtown,  Connecticut,  takes  a different  approach  that  relies  on  teachers  in  planning 
professional  development  at  both  the  district  and  school  levels.  At  the  district  level,  there  is  a 
professional  development  committee  that  consists  of  two  co-chairs,  one  teacher  representative  from 
every  building  or  department,  a paraprofessional  representative,  parents,  an  assistant  superintendent, 
principals,  assistant  principals,  a special  educator,  a science  coordinator,  mathematics  coordinator, 
and  Eisenhower  coordinator.  The  committee  designs  a professional  development  plan  for  the  district, 
and  revises  it  based  on  input  from  a principal  focus  group.  But  in  this  district,  there  is  a building 
instructional  team  that  assesses  the  needs  of  teachers  at  each  school.  The  team,  which  consists  of 
teachers,  administrators,  parents,  and  school  board  members,  carries  out  needs  assessments  in  the 
school,  which  includes  reviewing  test  results  by  grade  level,  and  develops  school-specific  plans.  The 
principal  sets  professional  development  goals  for  individual  teachers,  based  in  part  on  the  analyses  of 
the  committee. 

Weller,  Kentucky,  also  maintains  a balance  between  district  and  school  levels  in  planning 
professional  development  activities.  The  district  convenes  a task  force  that  meets  periodically 
throughout  the  year  to  monitor  professional  development  needs  and  oversee  progress.  The  task  force 
has  one  representative  per  school  (either  a teacher  or  administrator).  In  addition,  each  school 
develops  a consolidated  plan  that  identifies  the  school’s  professional  development  needs,  and 
outlines  ways  to  address  them  and  track  progress.  The  principal  and  a committee  of  teachers  develop 
the  school  plan. 

These  case  findings  illustrate  how  some  districts  rely  on  teachers,  in  different  ways,  as  active 
participants  on  committees  that  shape  professional  development  activities. 

Meeting  the  Needs  of  Teachers  vs.  Schools:  The  80-20  Rule 


As  our  survey  and  case  data  indicate,  districts  vary  in  their  emphasis  on  planning  at  the 
district,  cluster,  and  school  levels,  and  in  how  they  involve  teachers  in  planning.  This  is  a critical 
issue  of  implementation  to  understand,  because  the  Eisenhower  legislation’s  “80-20”  rule  appears  to 
focus  on  the  importance  of  planning  for  Eisenhower-assisted  activities  by  school-level  staff.  The  “80- 
20”  rule  states  that 

Each  local  educational  agency  that  receives  funds  under  this  part  for  any  fiscal  year  shall  use 
not  less  than  80  percent  of  such  funds  for  professional  development  of  teachers,  and,  where 
appropriate,  administrators,  and,  where  appropriate,  pupil  services  personnel,  parents,  and 
other  staff  of  individual  schools  in  a manner  that  is  determined  by  such  teachers  and  staff;  to 
the  extent  practicable,  takes  place  at  the  individual  school  site;  and  is  consistent  with  the  local 
educational  agency's  application  under  section  2208,  any  school  plan  under  part  A of  title  I, 
and  any  other  plan  for  professional  development  carried  out  with  Federal,  State,  or  local 
funds  that  emphasizes  sustained,  ongoing  activities  (Section  2210(a)(l)(A)(B)(C)). 
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Our  exploratory  case  studies  revealed  that  there  is  a lack  of  clarity  about  the  meaning  of  this 
rule  (Birman,  Reeve,  & Sattler,  1998).  We  found  that  Eisenhower  coordinators  are  unsure  of 
whether  the  rule  refers  to  involvement  of  teachers  from  multiple  schools  in  planning  professional 
development  at  the  district  level,  or  to  involvement  of  teachers  in  planning  professional  development 
with  others  in  their  own  schools,  or  both.  Teacher  involvement  at  the  district  level  can  help  to  ensure 
that  topics  and  learning  activities  in  professional  development  programs  address  areas  of  knowledge 
and  skills  that  are  relevant  to  teachers  district-wide.  This  can  be  particularly  important  for  teachers 
who  may  be  the  only  ones  teaching  a particular  subject  in  their  school  (e.g.,  physics  teachers  in  small 
high  schools).  At  the  same  time,  teacher  participation  in  planning  at  the  school  level  and 
professional  development  that  takes  place  at  the  school  allow  for  more  coherent  professional 
development  that  is  closely  tied  to  the  needs  of  teachers  in  a particular  school,  and  thus  potentially 
more  relevant  to  classroom  practice.  Some  believe  that  effects  on  teaching  practice  are  more  likely 
when  there  is  a critical  mass  of  teachers  in  a school  that  are  all  trying  to  improve  their  practice  in  the 
same  way.  In  Chapter  3,  we  reported  the  importance  of  collective  participation  to  the  perceived 
benefits  of  the  professional  development. 

Our  national  survey  of  Eisenhower  coordinators  suggests  that  districts  interpret  the  “80-20” 
rule  in  very  different  ways.  Although  many  district  coordinators  report  that  they  understand  the 
requirement  “very  well”  (42  percent)  or  “adequately”  (36  percent),  they  do  not  all  interpret  it 
similarly  (data  not  shown).  In  our  survey,  we  asked  Eisenhower  coordinators  to  explain  how  they 
implement  the  80-20  requirement  in  their  district,  and  common  responses  include  the  following 
approaches: 

♦ The  district  conducts  a needs  assessment  to  determine  teachers’  professional  development 
needs  and  provides  professional  development  to  meet  the  needs  identified. 

♦ At  least  80  percent  of  Eisenhower  funds  are  allocated  directly  to  the  school  level  (in  some 
districts,  schools  must  apply  for  their  share  of  the  funds). 

♦ A committee  of  teachers  and  administrators  identifies  professional  development  needs  in 
the  district  and  plans  activities  to  meet  those  needs. 

♦ Most  or  all  Eisenhower-assisted  professional  development  activities  must  take  place  at 
the  school  site. 

♦ Teachers  or  schools  select  professional  development  based  on  their  own  needs,  in  some 
cases  selecting  from  a menu  of  activities  provided  by  the  district. 

It  is  clear  from  this  list  that  districts  interpret  the  80-20  rule  in  a variety  of  ways.  Some 
districts  have  devolved  funding  and  all  decisions  about  Eisenhower  professional  development  to 
schools,  while  other  districts  have  responded  by  continuing  to  plan  professional  development 
activities  at  the  district  level,  with  a variety  of  types  of  teacher  input  (e.g.,  through  committees,  needs 
assessments,  etc.).  In  some  districts,  the  provision  that  Eisenhower-assisted  activities  “be  determined 
by  school-level  staff’  means  simply  that  teachers  determine  their  own  professional  development 
activities  by  selecting  activities  from  a district-determined  menu.  Finally,  some  districts  apparently 
interpret  the  rule  to  apply  to  the  location  of  professional  development  activities  themselves.  As 
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Other  approaches  to  the  80-20  rule,  however,  illustrate  more  confusion  about  its  meaning.  In 
some  districts,  the  rule  is  interpreted  as  follows: 

♦ 80  percent  of  funds  support  activities  in  mathematics  and  science  with  the  remaining  20 
percent  supporting  activities  in  other  subject  areas. 

♦ 80  percent  of  funds  are  used  to  pay  for  professional  development  activities  with  the 
remaining  20  percent  used  to  pay  for  program  administration. 

In  addition,  in  several  districts,  coordinators  simply  say  they  do  not  know  how  to  interpret  the 
rule.  Clearly,  the  current  80-20  provision  does  not  provide  clear  direction  to  the  nation’s  school 
districts.  Perhaps  one  reason  that  the  interpretations  of  the  80/20  rule  are  so  varied  is  that  it  mixes 
together  very  different  elements — the  level  of  planning  for  professional  development,  and  the 
location  of  the  professional  development  itself.  Our  case  studies  illustrate  that  teacher  involvement  in 
planning  professional  development  can  be  independent  from  locating  professional  development  at  the 
school,  or  having  staff  from  the  school  participate  together  in  professional  development  activities. 

In  a number  of  our  case  districts,  Eisenhower-assisted  professional  development  is  planned  at 
the  district  level,  but  the  district  emphasizes  approaches  to  professional  development  that  occurred  at 
the  school  level.  In  Commuteville,  Virginia,  a school-board-appointed  committee  conducts  a needs 
assessment  and  makes  recommendations  for  Eisenhower-assisted  activities.  The  committee  is 
composed  of  teachers,  parents,  students,  administrators,  and  community  representatives.  While  this 
planning  occurs  at  the  district  level,  Eisenhower  funds  support  two  types  of  professional 
development  that  take  place  in  schools.  First,  a Colleague  Teacher  Program  attempts  to  meet  the 
needs  of  first-year  teachers  by  pairing  up  new  teachers  with  experienced  teachers  who  offer  the  new 
teachers  guidance,  assistance,  and  support  over  the  course  of  a the  school  year.  Second,  the  district 
trains  promising  teachers  as  “lead  teachers”  in  mathematics  and  science.  These  teachers  serve  as 
resources  and  mentors  to  other  teachers  in  their  schools.  The  district’s  long-term  goal  is  to  have  lead 
teachers  for  mathematics  and  science  in  each  school. 

In  Boonetown,  Kentucky,  professional  development  occurs  largely  at  the  school,  while 
planning  the  strategy  for  professional  development  is  a shared  activity  of  the  district  Eisenhower 
coordinator  and  all  of  the  principals  in  the  district.  In  Boonetown,  resource  teachers  are  assigned  to 
schools,  where  they  work  with  principals  to  identify  school  needs  for  professional  development.  The 
strategy  grew  out  of  a number  of  district-wide  committees  that  recommended  reliance  on  resource 
teachers.  The  Eisenhower  coordinator  brought  this  idea  to  a meeting  of  principals,  who  arranged  to 
fund  the  resource  teachers  with  school  shares  of  professional  development  and  instructional  funds,  in 
addition  to  Eisenhower  funds.  The  district  Eisenhower  coordinator  has  primary  responsibility  for 
planning  Eisenhower-assisted  professional  development,  although  she  relies  heavily  on  all  of  the 
district’s  principals,  as  a group,  in  making  the  plans. 


7 The  Department  of  Education’s  Office  of  the  General  Counsel  communicated  its  interpretation  of  the  80-20  rule 
during  the  preparation  of  the  first  report  of  this  evaluation  in  September  1997,  and  again  during  the  review  of  this 
report  in  September  1999. 
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In  contrast  to  Commuteville  and  Boonetown,  other  case  districts  rely  heavily  on  individual 
schools  to  develop  plans  for  professional  development  activities.  In  Richmond,  New  York,  each 
school  may  apply  to  the  district  for  a “mini-grant”  of  $2,000  of  the  district’s  Eisenhower  funds.  But 
delegating  planning  to  the  school  level  does  not  necessarily  result  in  professional  development 
activities  that  are  embedded  in  teachers’  daily  activities,  or  even  in  activities  that  occur  at  the  school 
itself.  The  elementary  schools  in  the  district  generally  use  their  grants  for  one-day  workshops  led  by 
outside  consultants.  The  district’s  high  school,  which  receives  the  same  amount  of  funding  as  the 
elementary  schools,  despite  its  larger  enrollment,  uses  its  mini-grant  to  allow  teachers  to  judge 
science  fairs,  attend  conferences,  and  attend  “professional  development  activities  of  their  choice,” 
according  to  the  district  Eisenhower  coordinator. 

Both  the  literature  and  Eisenhower  legislation  support  teacher  participation  in  planning 
professional  development  activities,  and  professional  development  that  involves  all  teachers  in  a 
school.  The  data  from  our  teacher  activity  survey  (reported  in  Chapter  3 of  this  report)  support  the 
value  of  teachers  participating  in  professional  development  activities  with  others  from  their  schools, 
departments,  or  grade.  However,  our  case  data  illustrate  that  the  level  at  which  planning  occurs,  and 
teachers’  involvement  in  it,  are  not  necessarily  related  to  the  occurrence  of  professional  development 
at  the  school  itself,  who  participates,  or  the  quality  of  the  activities.  In  other  words,  planning  for 
professional  development  at  the  school  level,  or  occurrence  of  the  professional  development  activity 
at  the  school  site,  does  not  necessarily  mean  that  the  professional  development  involves  the  collective 
participation  of  all  teachers  or  groups  of  teachers  at  the  school. 

Summary:  Involvement  of  School  Level  Staff  in  Planning  for  Professional 
Development 

Clearly,  the  participation  of  school  staff  in  planning  professional  development  is  an  important 
emphasis  of  the  Eisenhower  legislation  and  the  professional  development  literature.  Participation  in 
professional  development  that  is  embedded  in  the  daily  life  of  teachers  at  their  schools  also  is 
emphasized  in  both  the  literature  and  the  legislation.  Our  national  data  indicate  that  although  teachers 
are  virtually  always  involved  in  planning  in  some  way  at  the  district,  school,  and/or  cluster  level, 
teachers  participate  on  a formal  basis  in  committees  at  about  two-thirds  of  the  levels  at  which 
planning  takes  place.  Furthermore,  while  teacher  participation  in  planning  professional  development 
may  be  valuable  in  communicating  what  is  important  to  teachers,  it  appears  not  to  be  related  to  the 
occurrence  of  professional  development  at  the  school  itself. 

The  intent  of  the  legislation’s  80-20  rule  is  to  provide  guidance  about  the  involvement  of 
teachers  in  planning  professional  development,  and  the  location  of  professional  development 
activities.  However,  the  legislative  language  allows  for  a large  variety  of  interpretations.  In  open- 
ended  responses,  districts  demonstrate  a large  variance  in  how  they  interpret  the  80-20  rule.  On  the 
one  hand,  this  rule  could  be  interpreted  as  reflecting  the  perception  of  both  Congress  and  professional 
development  experts  that  decisions  about  professional  development  are  best  made  at  the  school  level. 
On  another  hand,  the  rule  could  be  interpreted  as  calling  for  school-level  professional  development  at 
the  school  site. 

Thus,  while  districts  seem  to  be  fulfilling  the  requirements  of  the  legislation  by  involving 
teachers  in  planning  and  focusing  some  planning  at  the  school  level,  the  extent  to  which  these  actions 
are  actually  fulfilling  Congress’  intent  is  unclear.  Most  of  the  interpretations  districts  have  made  of 
the  80-20  rule  seem  to  comply  with  the  law,  at  least  as  interpreted  by  the  Department  of  Education. 
However,  Congress’  intent  in  this  provision  has  not  been  well  understood  by  states  or  by  districts.  It 


is  possible  that  in  this  provision,  the  Congress  intended  to  ensure  that  the  needs  of  teachers,  as 
identified  by  teachers  themselves,  were  being  taken  into  account  in  the  planning  of  professional 
development.  If  this  was  the  Congress’  intent,  it  seems  largely  to  have  been  met,  though  at  times 
without  teachers’  active  participation  in  a planning  process  (through  participation  in  formal  planning 
committees). 

It  is  also  possible  that,  in  addition  to  serving  individual  teachers’  needs,  the  Congress 
intended  that  EisCnhower-assisted  professional  development  serve  school  needs  in  the  service  of 
school-wide  goals.  If  this  was  what  the  Congress  intended,  then  our  data  suggest  that  districts  could 
place  more  emphasis  on  school-level  planning,  and  embedding  professional  development  in  the  daily 
lives  of  teachers.  Research  on  professional  development  suggests  that  meeting  the  needs  of 
individual  teachers  and  meeting  school-wide  needs  are  both  important  goals  of  professional 
development.  The  Department  of  Education  and  the  Congress  may  want  to  consider  how  to  make 
both  of  these  goals  clear  in  the  upcoming  reauthorization  of  the  Elementary  and  Secondary  Education 
Act. 

Differences  in  Management  and  Operation  of  Eisenhower- 
Assisted  Activities  by  District  Poverty  and  Size 


Throughout  this  chapter  we  report  differences  in  district  management  and  operation 
according  to  the  district’s  poverty  level  and  the  number  of  teachers  in  the  district.  A synthesis  of 
these  results  indicates  that  there  is  more  variation  by  district  size  than  by  poverty  level.  In  general, 
however,  large  districts  and  high-poverty  districts  perform  better  on  these  measures  than  small 
districts  and  low-  poverty  districts. 

Specifically,  we  find  that,  compared  to  low-poverty  districts,  high-poverty  districts: 

♦ co-fund  more  with  NSF  and  ED  programs,  and 

♦ employ  more  continuous  improvement  efforts. 

We  find  that,  compared  to  small  districts,  large  districts: 

♦ are  more  aligned  with  state  and  district  standards  and  assessments, 

♦ co-fund  more  with  NSF  and  ED  programs,  and 

♦ employ  more  continuous  improvement  efforts. 

Consortia  also  co-fund  and  use  continuous  improvement  efforts  significantly  more  than 
smaller  districts  do.  We  also  find  that  both  large  and  small  districts  have  significantly  more  school- 
level  planning  than  consortia.  Teacher  participation  in  planning  is  the  only  element  of 
implementation  that  does  not  vary  significantly  by  either  district  poverty  or  district  size. 

These  findings,  taken  together,  suggest  several  conclusions.  First,  large  districts  may  be 
more  aligned,  and,  along  with  consortia,  may  co-fund  more,  and  use  more  continuous  improvement 
methods  because  they  have  a greater  need  to  integrate  professional  development  with  other  reforms. 
Large  districts  and  consortia  may  have  an  increased  need  for  organization  and  integration  since  they 
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are  more  likely  to  have  a large  number  of  iniatives  and  reforms  and  more  money  from  other  federal 
programs,  and  because  of  the  larger  number  of  teachers  that  they  serve.  This,  in  turn,  would  also 
create  the  need  for  more  systematic  monitoring  and  evaluation  of  their  professional  development 
(and  other  reform)  efforts.  Another  reason  that  they  outperform  smaller  districts  on  these  dimensions 
may  be  that  larger  districts  and  consortia  are  likely  to  have  a better  infrastructure  for  coordination 
among  district-level  staff  who  serve  in  professional  development  roles. 

Similarly,  high-poverty  districts  may  co-fund  more  because  they  are  more  likely  to  have 
multiple  programs  operating  that  share  a focus  on  targeting  teachers  of  special  populations  of 
students.  Similarly,  districts  with  more  students  in  poverty  may  conduct  more  continuous 
improvement  efforts  because  they  are  likely  to  be  receiving  funds  from  federal  programs  that  have 
requirements  similar  to  Eisenhower’s  for  the  use  of  indicators,  needs  assessments,  evaluation,  and 
guidance. 

The  finding  that  school-level  planning  is  less  common  in  consortia  than  in  either  districts  or 
schools  may  be  because  the  main  purpose  of  consortia  is  to  foster  across-district  planning,  rather  than 
planning  across  schools  or  within  individual  schools.  Single  districts  that  do  not  have  the  added  issue 
of  across-district  coordination  may  have  more  latitude  to  focus  planning  at  the  individual  school 
level. 

These  results  show  that  alignment,  co-funding,  continuous  improvement,  and  teacher 
involvement  in  planning  vary  across  districts.  The  next  section  informs  our  understanding  of  how 
these  four  factors  affect  the  quality  of  professional  development. 


The  Relationship  of  District  Management  to  Features  of 

Professional  Development 


In  this  chapter  so  far,  we  have  described  several  aspects  of  the  district  management  and 
operation  of  Eisenhower-assisted  activities.  In  Chapter  4,  we  examined  the  characteristics  of  district 
portfolios  of  professional  development.  Here,  we  draw  together  our  results  in  these  two  chapters  by 
analyzing  the  extent  to  which  district  management  practices  are  related  to  particular  structural  and 
core  features  of  district  Eisenhower  professional  development  portfolios. 

As  described  in  Chapter  4,  we  developed  measures  of  Eisenhower  portfolio  characteristics. 
Specifically,  they  are  (1)  the  percent  of  the  districts’  Eisenhower  participations  in  reform  activities; 
(2)  the  average  time  span  of  the  districts’  activities,  both  reform  and  traditional;  (3)  the  extent  to 
which  activities  have  collective  participation — participation  by  whole  schools  or  groups  of  teachers 
(e.g.,  all  teachers  from  the  same  grade  or  department);  (4)  the  number  of  opportunities  for  active 
learning  offered  in  in-district  workshops  or  institutes;  and  (5)  the  amount  of  emphasis  the  district 
places  on  targeting  professional  development  activities  to  teachers  of  special  populations  of  students 
(e.g.,  teachers  from  Title  I schools). 

In  Chapter  4,  we  noted  that  these  measures  differ  by  district  poverty  and  size.  Here,  we  take 
this  analysis  of  portfolio  characteristics  one  step  further  by  presenting  a model  that  describes  how  the 
district’s  role  in  shaping,  implementing,  and  planning  professional  development,  described  earlier  in 
this  chapter,  is  related  to  the  structural  and  core  features  of  the  district’s  portfolio  of  activities. 
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To  analyze  how  the  various  components  of  the  legislation — those  addressing  the  district’s 
role  in  design,  quality,  implementation,  and  targeting  of  Eisenhower-assisted  professional 
development  activities — are  associated  with  each  other,  we  developed  an  explanatory  model,  shown 
in  Exhibit  5.11.  This  model  takes  each  major  component  of  the  legislation,  as  measured  by  variables 
constructed  from  our  survey  of  Eisenhower  coordinators  and  described  in  this  chapter  and  Chapter  4, 
and  examines  its  relationship  to  several  other  components. 

The  model  is  an  implied  logic  model,  in  that  we  hypothesize  a sequence  of  events. 
Specifically,  as  Exhibit  5. 1 1 depicts,  we  assume  that  districts  first  build  a vision  of  professional 
development  through  alignment  and  co-funding,  then  implement  and  monitor  the  vision  through 
planning  and  continuous  improvement  efforts.  These  actions  then  result  in  particular  features  of  the 
district  portfolio  of  professional  development,  such  as  the  percent  of  teachers  in  reform  types,  the 
average  duration  of  activities,  the  degree  of  collective  participation,  opportunities  for  active  learning, 
and  the  district’s  targeting  practices.  It  should  be  emphasized,  however,  that  components  of  the 
system  are  likely  interactive,  and  may  occur  simultaneously.  For  example,  a reform-oriented  district 
may  practice  superior  vision-building  and  implementation,  and  design  activities  with  more  high- 
quality  components  and  more  targeting,  all  at  the  same  time,  because  of  the  district’s  orientation 
toward  reform.  Our  data  are  not  longitudinal,  so  we  cannot  test  the  causal  ordering.  We  can, 
however,  identify  the  strength  of  relationships  among  variables.  We  suggest  a logic  of  events  to  help 
to  explain  how  the  process  of  designing  and  implementing  district-provided  development  might 
work;  but  our  model  should  not  be  considered  to  exclude  the  possibility  of  two-way  effects  or  an 
alternative  temporal  ordering. 

We  use  ordinary  least  squares  regression  (OLS)  to  analyze  the  paths  (or  associations) 
between  variables.  Only  relationships  that  are  significant  at  the  .05  level  are  reported.  Since 
contextual  factors  may  influence  the  design  and  implementation  of  district  portfolios,  we  have 
included  several  district  characteristics  as  control  variables  in  our  model:  district  poverty  level, 
consortium  status,  the  log  of  the  number  of  teachers,8  the  interaction  of  the  log  of  the  number  of 
teachers  and  consortium  status,9  and  cluster  status.  (For  a detailed  description  of  all  of  the  variables 
in  the  model,  see  Appendix  G.) 

As  Exhibit  5.1 1 shows,  co-funding  is  the  strongest  predictor  of  the  features  of  district 
portfolios  of  Eisenhower-assisted  professional  development.  It  is  related  directly  to  increased 
targeting  (b=.  17), 10  a higher  percent  of  teacher  participations  in  reform  types  of  professional 
development  (b=.15),  and  more  collective  participation  (b=.14),  and  it  is  indirectly  related  to  more 
opportunities  for  active  learning  and  increased  targeting  through  its  relationship  to  increased 
continuous  improvement  efforts  (b=.  1 6)  and  more  teacher  participation  in  planning  (b=.  1 6). 


8 We  expect  that  the  same  increase  in  district  size  has  a smaller  effect  on  implementation  and  outcome  factors  as  size 
increases.  For  example,  a 100  teacher  increase  in  district  size  from  100  to  200  teachers  would  have  more  of  an 
effect  on  implementation  and  outcome  factors  than  a 100  teacher  increase  in  district  size  from  1,000  to  1,100 
teachers.  Taking  the  log  of  district  size  allows  measured  changes  in  the  dependent  variable  to  be  associated  with 
proportional  increases  in  the  number  of  teachers  in  a district. 

9 The  effect  of  size  may  differ  for  consortia  and  individual  districts.  Measuring  the  interaction  of  the  log  of  district 
size  and  consortium  status  allows  us  to  take  this  into  account. 

10  B represents  the  standardized  beta  coefficient,  or  the  standardized  regression  coefficient,  which  indicates  the 
strength  of  the  relationship  between  the  two  variables.  For  example,  the  beta  of  .17  for  the  relationship  between  co- 
funding and  targeting  means  that  for  every  one  standard  deviation  increase  in  co-funding,  there  is  a .17  standard 
deviation  increase  in  targeting.  The  arrow  from  co-funding  to  targeting  indicates  that  targeting  was  regressed  on  co- 
funding. 
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Coordination  in  terms  of  working  with  schools  and  professional  development  providers  proved 
unimportant  in  our  model,  but  alignment  is  significantly  related  to  implementation,  and  structural  and 
core  features  of  professional  development.  Alignment  predicts  more  participations  in  reform  types  of 
professional  development  (b=.12),  which  in  turn  is  associated  with  a longer  span  (b=.40),  and  also 
more  continuous  improvement  (b=.  16).  These  results  support  the  notion  that  building  a vision  of 
professional  development  through  alignment,  and  having  a critical  mass  of  funds  available,  made 
possible  through  co-funding,  are  instrumental  factors  in  fostering  the  provision  of  high-quality 
professional  development  activities.  Further,  having  activities  aligned  with  state  and  district 
standards  and  assessments  may  indicate  that  districts  are  providing  guidance  and  using  data  for 
continuous  improvement  efforts. 

Although  this  analysis  demonstrates  the  importance  of  alignment  and  co-funding,  which  are 
emphasized  in  the  Eisenhower  legislation,  we  still  do  not  know  as  much  as  we  could  about  the  extent 
to  which  districts  practice  alignment  and  co-funding,  or  the  processes  through  which  they  engage  in 
these  practices.  Developing  a deeper  understanding  of  how  alignment  and  co-funding  work  would 
help  us  to  understand  their  link  to  other  implementation  efforts,  and  ultimately,  to  the  provision  of 
high-quality  professional  development  activities. 

Continuous  improvement  is  another  variable  that  is  associated  with  outcomes  that  are 
emphasized  by  the  legislation,  but  whose  actual  operation  remains  unclear.  Continuous  improvement 
efforts,  in  terms  of  providing  guidance,  needs  assessments,  and  evaluation,  have  a moderate 
association  with  both  increased  opportunities  for  active  learning  (b=.20)  and  increased  targeting 
(b=.17).  Our  analyses  of  the  sub-scales  that  comprise  continuous  improvement,  reported  earlier  in 
this  chapter,  show  several  potentially  important  findings:  1)  districts’  use  of  indicators  is  not 
prevalent,  and  case  studies  suggest  that  for  those  that  do  use  indicators,  their  use  is  somewhat 
perfunctory;  2)  evaluative  methods  do  not  include  linking  professional  development  with  teacher 
actions  or  student  outcomes;  and  3)  many  districts  offer  particular  types  of  guidance  and  support  to 
schools  and  professional  development  providers,  but  offer  little  guidance  related  to  the  use  of  data. 
This  indicates  that  while  districts  play  a role  in  guiding  schools  and  professional  development 
providers,  districts  may  lack  the  capacity  for  sophisticated  use  of  data  in  decision-making,  planning, 
and  evaluation. 

In  terms  of  planning,  Exhibit  5.11  shows  that  district-level  planning  (as  opposed  to  school- 
level  planning)  is  related  to  activities  of  longer  span  (b=.l  1),  and  teacher  participation  in  planning  is 
related  to  activities  with  more  opportunities  for  active  learning  (b=.20)  and  more  targeting  of  teachers 
of  special  populations  of  students  (b=.  16).  These  relationships  support  our  findings  reported  earlier 
in  this  chapter.  High-quality  professional  development  can  be  planned  at  any  level,  and  planning  at 
the  district  level  is  not  the  same  as  implementation  at  the  district  level.  As  our  case  studies  show, 
there  are  examples  of  both  high-quality  professional  development  planned  at  the  district  level  and 
low-quality  professional  development  planned  at  the  school  level.  This  reinforces  the  need  to  clarify 
the  80-20  rule,  to  determine  Congress’  intent  in  encouraging  school-level  planning  and 
implementation  of  professional  development  activities. 
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Relationship  of  District  Management  to  Features  of  Professional  Development 
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Summary  and  Conclusions 


In  this  chapter,  we  described  how  districts  manage  and  operate  Eisenhower-assisted 
professional  development  activities.  We  examined  how  districts  build  a vision  for  professional 
development  through  alignment  and  coordination,  and  how  they  implement  that  vision,  through 
continuous  improvement  efforts  and  through  planning  at  different  levels  with  different  types  of 
teacher  involvement.  We  also  examined  how  these  management  practices  and  operations  differ  in 
districts  of  different  poverty  levels  and  sizes,  and  we  present  a model  of  how  these  practices  shape 
the  quality  of  Eisenhower-assisted  professional  development  activities.  Several  of  the  findings  of 
these  analyses  have  important  implications  for  the  Eisenhower  Professional  Development  Program. 

First,  districts  are  much  more  likely  to  have  their  professional  development  aligned  with 
standards  than  with  assessments.  This  suggests  that  many  districts  may  not  yet  have  their 
assessments  aligned  with  district  and/or  state  standards.  If  one  major  goal  of  the  Eisenhower 
legislation  is  to  fit  professional  development  into  the  framework  of  other  reforms  in  the  district,  it 
may  be  that  districts  have  considerable  work  to  do  to  establish  connections  and  alignment  across 
standards  and  assessments  before  this  is  possible. 

Second,  we  find  that  co-funding  plays  an  important  role  in  the  implementation,  management, 
structure,  and  core  features  of  Eisenhower-assisted  professional  development.  Our  data  show  that  co- 
funding most  often  occurs  with  other  programs  focused  on  mathematics  and  science,  and  that  other 
types  of  coordination  can  be  less  substantive  and  meaningful  in  shaping  professional  development. 
This  highlights  the  importance  of  having  subject-area  focus  in  common  in  order  to  promote  co- 
funding. Also,  gaining  a more  comprehensive  understanding  of  the  dynamics  involved  in  combining 
funding  streams  with  other  programs  would  help  us  focus  on  how  this  process  results  in  districts 
being  better  able  to  provide  professional  development  that  meets  the  high  standards  of  quality 
outlined  in  the  provisions  of  the  Eisenhower  legislation. 

Third,  continuous  improvement  efforts  are  related  to  certain  portfolio  features,  such  as 
greater  opportunities  for  active  learning  and  targeting,  but  these  efforts  are  relatively  rare. 

Eisenhower  coordinators  are  least  likely  to  report  using  data-driven  continuous  improvement 
methods,  such  as  applying  indicators  to  professional  development  or  linking  teacher  practices  with 
student  achievement  to  evaluate  professional  development  activities.  Perhaps  more  emphasis  on 
these  important  and  more  rigorous  methods  of  continuous  improvement  would  increase  the  quality  of 
these  efforts,  and  as  a result,  improve  the  quality  of  the  professional  development  activities  that 
districts  provide. 

Lastly,  our  analyses  of  differences  according  to  district  poverty  level  and  district  size  suggest 
that  there  are  certain  advantages  that  large  districts  have  over  small  districts,  and  that  high-poverty 
districts  have  over  low-poverty  districts,  in  shaping,  planning,  and  implementing  Eisenhower-assisted 
professional  development  activities.  It  is  likely  that  the  presence  of  more  federal  dollars,  more  staff 
(and  thus  more  need  to  coordinate),  and  better  infrastructure  for  communication  and  coordination  all 
contribute  to  these  differences.  Learning  more  about  the  conditions  that  facilitate  better 
implementation  and  planning  in  certain  types  of  districts  would  help  to  identify  possible  added 
supports  that  some  districts  need;  it  may  also  suggest  particular  conditions  or  processes  that  could  be 
established  or  required  by  the  Eisenhower  legislation  in  order  to  develop  capacity  in  other  districts. 
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CHAPTER  6 


STATE  AGENCY  FOR  HIGHER  EDUCATION  GRANTEES: 
FEATURES  AND  MANAGEMENT  OF  EISENHOWER- 
ASSISTED  ACTIVITIES  IN  INSTITUTIONS  OF  HIGHER 
EDUCATION  AND  NONPROFIT  ORGANIZATIONS 


The  Eisenhower  legislation  includes  a component  of  the  program  for  professional 
development  projects  sponsored  by  state  agencies  for  higher  education  (SAHEs).  This  component  of 
the  Eisenhower  program  operates  separately  from  the  component  that  supports  district-sponsored 
activities.  The  legislation  allocates  16  percent  of  Eisenhower  funding  for  the  SAHE  component  of 
the  Eisenhower  program.  Unlike  the  remaining  84  percent  of  Eisenhower  funding,  which  is 
distributed  by  a formula  through  State  Educational  Agencies  (SEAs)  to  local  educational  agencies 
(LEAs),  SAHEs  sponsor  competitions  within  their  states  and  make  awards  to  organizations  that 
apply.  Grants  are  awarded  primarily  to  institutions  of  higher  education  (IHEs),  such  as  colleges  and 
research  universities,  and  nonprofit  organizations  (NPOs),  such  as  museums  and  libraries.  In 
sponsoring  these  competitions,  each  SAHE  develops  priorities  and  establishes  criteria  for  awarding 
the  grants.  The  priorities,  guidelines,  and  criteria  are  based  to  some  extent  on  the  state  plan  to 
improve  teaching  and  learning,  which  the  legislation  requires  the  SEA  to  develop  in  conjunction  with 
the  SAHE,  as  part  of  the  application  process.  In  the  plan,  SEAs  and  SAHEs  describe  the  process  and 
results  of  their  assessment  of  the  needs  of  their  teaching  force,  and  how  the  activities  that  the  state 
intends  to  provide  will  address  teachers’  needs,  including  recruitment,  pre-service  and  induction  and 
continuing  through  the  provision  of  in-service  activities,  as  well  as  how  the  professional  development 
plan  incorporates  standards  and  indicators  and  provides  for  the  needs  of  teachers  of  special 
populations  of  students. 

The  SAHE  component  of  the  Eisenhower  program  takes  advantage  of  the  prominent  role  of 
IHEs  in  preparing  the  nation’s  teachers.  This  component  is  designed  to  encourage  IHEs  and  NPOs  to 
provide  high-quality  in-service  and  pre-service  professional  development  that  is  consistent  with  state 
standards  and  reforms,  and  to  foster  closer  ties  between  higher  education  institutions  and  elementary 
and  secondary  education  agencies. 

SAHE-sponsored  IHE/NPO  Eisenhower  projects  (referred  to  as  “SAHE  grantees”)  are 
subject  to  the  same  stipulations  regarding  quality  that  apply  to  district  activities — requirements  for 
activities  to  be  in  core  subject  areas,  mainly  in  mathematics  and  science,  but  also  including  other  core 
academic  subject  areas,  and  to  be  sustained,  intensive,  and  innovative.  Specifically,  the  legislation 
says  that  the  grants,  contracts,  and/or  cooperative  agreements  established  with  the  SAHE  grantee 
shall  be  for 

Professional  development  activities  in  the  core  academic  subjects  that  contribute  to  the  State 
plan  for  professional  development  (Section  221 1(a)(1)(A));  developing  and  providing 
assistance  to  local  educational  agencies,  and  the  teachers  and  staff  of  each  such  agency,  for 
sustained  high-quality  professional  development  activities  (Section  221 1(a)(1)(B));  and 
improving  teacher  education  programs  in  order  to  promote  further  innovation  in  teacher 
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education  programs  within  an  institution  of  higher  education  and  to  better  meet  the  needs  of 
the  local  educational  agencies  for  well-prepared  teachers  (Section  221 1(a)(1)(C)). 

The  law  also  describes  several  types  of  activities  that  SAHE  grantees  are  to  provide,  including 

Sustained  and  intensive  high-quality  professional  development  for  teams  of  teachers,  or 
teachers,  and,  where  appropriate,  pupil  services  personnel  and  administrators  from  individual 
schools  or  school  districts  (Section  221 1(b)(1));  other  sustained  and  intensive  professional 
development  activities  related  to  achievement  of  the  State  plan  for  professional  development 
(Section  221 1(b)(2));  and  pre-service  training  activities  (Section  221 1(b)(3)). 

The  SAHE  portion  of  the  law  also  includes  specific  coordination  requirements  for  grantees, 
namely,  that  they  “work  in  conjunction  with  a local  educational  agency”  (Section  221  l)(a)(l))  by 
“entering  into  an  agreement  with  an  LEA  to  provide  sustained,  high-quality  professional 
development”  (Section  221 1(3)). 

In  addition  to  these  specific  requirements  that  define  the  quality  of  SAHE-sponsored 
professional  development,  SAHE  grantees  must  be  responsive  to  the  general  purposes  of  the 
Eisenhower  Professional  Development  Program,  as  identified  in  the  legislation.  SAHE  grantees 
should  provide  professional  development  that,  for  example,  “includes  strong  academic  content  and 
pedagogical  components”  (Section  2002(2)(C));  “is  of  sufficient  intensity  and  duration”  (Section 
2002(2)(E));  and  “creates  an  orientation  toward  continuous  improvement  throughout  the  school” 
(Section  2002(2)(F)). 

Similarly,  the  law’s  general  goals  for  providing  professional  development  to  teachers  of 
students  of  diverse  needs  apply  to  SAHE  grantees.  These  provisions  stipulate  that  professional 
development  provided  under  the  Eisenhower  program  should  “incorporate  effective  strategies”  to 
meet  the  needs  of  teachers  of  “diverse  student  populations”  (Section  2002(2)(D)).  SAHE  grantees 
also  are  subject  to  the  general  provisions  of  the  law  that  state  that  professional  development  offered 
under  the  Eisenhower  program  should  be  “tied  to  challenging  State  content  standards  and  challenging 
State  student  performance  standards”  (Section  2002(2)(A)). 

SAHEs  are  responsible  for  designing  competitions  and  awarding  grants  to  IHE/NPO 
professional  development  projects  that  meet  the  criteria  outlined  in  the  legislation.  In  this  chapter, 
we  describe  the  extent  to  which  SAHE  grantees  meet  the  requirements  of  the  legislation.  We 
provide  information  about  the  SAHE-grantee  characteristics  and  practices  that  shape  their  activities: 
the  structural  and  core  features  of  professional  development  activities;  recruitment  and  targeting  of 
teachers  of  special  populations  of  students;  alignment  with  state  and  district  standards  and 
assessments  and  coordination  with  districts;  and  the  use  of  continuous  improvement  mechanisms, 
such  as  indicators,  needs  assessments,  and  evaluation.  Exhibit  6.0  illustrates  how  this  chapter  fits  into 
the  conceptual  framework  of  the  entire  study. 


EXHIBIT  6.0 


Conceptual  Framework  for  This  Evaluation 


Data  Sources 

In  this  chapter,  we  use  our  survey  data  from  SAHE-grantee  project  directors.  During  the 
spring  of  1998,  we  conducted  telephone  interviews  with  a nationally  representative  sample  of  92 
SAHE  grantee  project  directors.  This  represents  a response  rate  of  87  percent.  To  develop  the 
sampling  frame,  we  contacted  all  50  SAHE  Eisenhower  coordinators  and  obtained  complete  lists  of 
projects  supported  during  the  1997-98  school  year.  We  sampled  SAHE  grantees  in  proportion  to  the 
size  of  their  Eisenhower  grant,  based  on  the  assumption  that  the  number  of  teachers  served  was 
proportional  to  the  size  of  the  grant.  After  collecting  information  on  the  number  of  teachers  who 
participated  in  SAHE-grantee  activities,  we  weighted  the  data  according  to  the  number  of  teachers 
actually  served  by  the  SAHE  grantees.1  As  a result,  our  data  are  representative  of  all  teachers 
participating  in  SAHE-grantee  projects.2 


1 The  correlation  between  grant  size  and  the  number  of  teachers  served  is  .6.  All  parameter  estimates  reported  in  the 
chapter  are  weighted.  Reported  p- values  and  the  standard  errors  on  which  they  are  based,  however,  do  not  reflect 
the  variance  in  weights  across  SAHE  grantees.  Analyses  that  take  this  element  of  the  complex  sample  design  into 
account  have  been  carried  out,  and  the  results  are  nearly  identical  to  those  reported  in  the  chapter. 

2 Throughout  the  chapter,  we  refer  to  the  percent  of  teachers  in  SAHE-grantee  projects;  but  our  data  actually 
represent  the  percent  of  teacher  participations  in  SAHE-grantee  projects.  These  two  figures  differ  based  on  the 
extent  to  which  the  same  teacher  participated  in  more  than  one  SAHE-grantee  activity.  For  clarity  of  presentation, 
and  because  multiple  participations  in  IHE/NPO  activities  are  minimal,  we  present  our  findings  in  terms  of  percent 
of  teachers. 
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We  asked  the  project  directors  questions  that  referred  to  the  time  period  from  July  1 through 
December  31,  1997  (the  same  time  period  for  which  district  coordinators  answered  questions,  as 
reported  in  Chapters  4 and  5 of  this  report).  All  data  presented  in  this  chapter  refer  to  that  time 
period,  unless  otherwise  noted.  SAHE  grantees  can  use  Eisenhower  funding  to  support  one  activity, 
or  several  activities.  On  our  survey,  a single  activity  is  defined  as  an  event  with  a common  group  of 
participants.3  Some  questions  on  the  survey  apply  to  all  SAHE-grantee  activities,  and  some 
questions  apply  only  to  the  grantee’s  “primary”  activity.4  For  questions  that  pertained  to 
characteristics  of  activities,  such  as  structural  and  core  features,  we  asked  SAHE-grantee  project 
directors  about  the  primary  professional  development  activity  that  they  support  with  Eisenhower 
funds.  Questions  that  refer  to  general  practices,  such  as  targeting  and  coordination,  apply  to  all  of  the 
grantee’s  Eisenhower-assisted  activities.  The  exhibits  in  this  chapter  indicate  when  the  data  apply 
only  to  the  IHE/NPO’s  primary  activity;  otherwise,  the  data  report  on  general  practices  that  apply  to 
all  of  the  SAHE  grantee’s  Eisenhower-assisted  activities.5  While  a SAHE  can  play  a key  role  in 
shaping  its  grantees’  projects  through  the  guidelines  and  criteria  it  establishes  for  the  grants 
competition,  in  this  chapter  we  focus  primarily  on  the  quality  and  operation  of  the  grantees’ 
Eisenhower  project.6 

We  use  our  case  studies  of  SAHE  grantees  to  explain  and  describe  particular  aspects  of 
professional  development  structure,  substance,  and  organization.  In  the  spring  and  summer  of  1998, 
we  conducted  open-ended  telephone  interviews  with  project  directors  at  six  IHEs  that  served  teachers 
in  our  case-study  districts.  We  conducted  in-depth  interviews  with  the  directors,  and  collected  their 
Eisenhower  grant  proposals,  and  descriptions  of  the  Eisenhower  activities  that  they  provided. 

The  data  reported  in  this  chapter  refer  to  Eisenhower-assisted  in-service  activities  that  are 
provided  by  SAHE  grantees.  Most  SAHE  grantees  conduct  pre-service  as  well  as  in-service 
professional  development  activities,  and  Eisenhower  funds  can  be  used  for  either  purpose.  We  focus 
in  this  report  on  in-service  activities,  since  nearly  all  of  SAHE-grantees  use  Eisenhower  funds  for 
this  purpose.7  Also,  while  districts  sometimes  may  use  IHE/NPOs  to  supply  Eisenhower-assisted 
professional  development  activities,  this  chapter  describes  only  the  IHE/NPO  professional 
development  activities  that  SAHEs  administer  through  the  Eisenhower  Professional  Development 
Program. 


3 For  example,  if  four  different  groups  of  teachers  attended  the  same  workshop  on  four  separate  occasions,  this 
would  count  as  four  activities.  But  if  one  group  of  teachers  attended  a workshop  and  a follow-up  event,  this  would 
count  as  one  activity. 

4 If  SAHE-grantee  project  directors  provided  more  than  one  Eisenhower-assisted  activity,  we  asked  them  to  identify 
one  as  their  primary  activity. 

5 Since  our  survey  asked  SAHE  grantees  to  describe  characteristics  of  their  primary  activity,  and  our  district  survey 
asked  district  coordinators  to  describe  characteristics  across  all  activities,  comparisons  of  responses  to  these 
questions  are  inappropriate.  However,  questions  about  targeting,  alignment,  coordination,  and  continuous 
improvement  efforts  refer  to  the  activities  of  both  SAHE  grantees  and  districts  overall;  therefore,  we  are  able  to 
compare  these  responses,  and,  where  such  comparisons  seem  instructive,  we  do  so. 

6 Due  to  the  small  number  of  grantees,  SAHEs  may  be  able  to  monitor  their  grantees’  projects  to  help  ensure  faithful 
implementation;  but  we  did  not  examine  the  SAHE’s  monitoring  role. 

7 In  1992-1993,  1993-1994,  and  1994-1995,  the  average  percent  of  Eisenhower  funds  that  State  Agencies  of  Higher 
Education  (SAHEs)  used  for  pre-service  activities  was  3 percent  in  each  year  (Donly  & Gutman,  1997);  for  1996- 
1997,  SAHEs  reported  that  14  percent  of  Eisenhower  funds  were  used  to  support  pre-service  professional 
development  (Celebuski  et  al.,  1998). 
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Organization  of  Chapter 


We  begin  the  chapter  by  describing  the  characteristics  of  the  SAHE  grantees  in  our  national 
sample.  We  then  provide  data  about  the  subject-area  and  structural  features  of  the  primary 
professional  development  activities  provided  by  our  sample  of  SAHE  grantees.  The  structural 
features  are  type  of  activity  (traditional  vs.  reform);  duration,  including  both  contact  hours  and  span 
across  days,  weeks,  months,  or  years;  and  collective  participation  (i.e.,  the  extent  to  which  activities 
are  designed  for  groups  of  teachers  or  whole  schools). 

The  next  section  reports  findings  on  core  features  of  activities,  specifically  the  strength  of  the 
content  focus  of  the  activity  and  the  number  of  active  learning  opportunities  provided  in  the  activity. 
Next,  we  focus  on  the  targeting  and  recruitment  of  teachers  of  special  populations  of  students.  Here 
we  examine  how  frequently  SAHE  grantees  target  teachers  of  special  populations  of  students,  how 
teachers  come  to  participate  in  SAHE-grantee  projects,  and  strategies  that  SAHE  grantees  use  to 
increase  teacher  participation. 

After  the  targeting  section,  we  analyze  the  extent  to  which  SAHE  grantees’  projects  are 
aligned  with  state  and  district  standards  and  assessments  and  coordinated  with  districts.  We  then 
report  on  the  continuous  improvement  efforts  of  SAHE  grantees — whether  they  use  state  and  district 
indicators  in  designing  professional  development,  and  whether  and  how  they  conduct  needs 
assessments  and  evaluate  their  projects. 

The  continuous  improvement  section  is  followed  by  a summary  and  analysis  of  findings 
concerning  significant  differences  on  key  variables  according  to  type  of  institution  and  departmental 
affiliation.  Throughout  the  chapter,  we  report  findings  according  to  the  type  of  institution  in  which 
the  Eisenhower  project  director  works,  and  the  department  with  which  the  project  director  is 
affiliated  because  these  factors  may  shape  the  structure  and  substance  of  professional  development 
activities  in  systematic  ways.  We  distinguish  two  types  of  institutions — research  universities  and 
universities  that  grant  doctorates;  and  all  other  types  of  colleges  and  universities  (e.g.,  two-year 
colleges,  private  liberal  arts  colleges,  and  institutions  that  grant  only  bachelor’s  or  master’s  degrees). 
We  distinguish  three  departmental  affiliations — mathematics  or  science  departments;  education 
departments;  and  “other”  departments  (e.g.,  administrative,  media,  or  broadcasting).  NPOs  are 
excluded  from  comparisons  of  institution  type  and  departmental  affiliation,  since  these  characteristics 
do  not  apply  to  them.  Otherwise,  all  analyses  combine  data  from  SAHE-supported  IHEs  and  NPOs. 

The  potential  significance  of  these  departmental  affiliation  distinctions  is  highlighted  in  the 
legislation,  which  defines  another  joint  effort  that  is  required  of  IHEs — between  the  IHE’s  school  of 
education  and  the  discipline  in  which  the  professional  development  is  being  provided.  The  law  states 
that 


Each  activity  assisted  under  this  section,  where  applicable,  shall  involve  the  joint  effort  of  the 
institution  of  higher  education’s  school  or  department  of  education,  if  any,  and  the  schools  or 
departments  in  the  specific  disciplines  in  which  such  professional  development  will  be 
provided  (Section  221 1(A)(4)). 

The  variation  in  project  directors’  educational  training  and  pedagogical  practices  across 
disciplines  and  in  different  types  of  IHEs  may  play  a role  in  shaping  the  professional  development 
activities  that  the  project  director  develops  and  provides.  To  examine  whether  or  not  the  structural 
and  core  features  and  implementation  of  professional  development  differs  according  to  the  type  of 
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IHE  or  the  departmental  affiliation  of  the  project  director,  we  include  both  of  these  factors  when  we 
analyze  the  main  variables  in  our  study  (i.e.,  participations  in  traditional  vs.  reform  activities, 
duration,  span,  collective  participation,  active  learning  opportunities,  content  focus,  targeting, 
alignment,  coordination,  and  continuous  improvement).  Although  a project  that  is  housed  in  a 
particular  department  may  be  administered  collaboratively  by  the  school  of  education  and  the  college 
or  school  in  which  the  mathematics/science  department  resides,  our  study  focuses  on  the 
departmental  affiliation  of  the  project  director.  Both  factors  are  included  in  all  analyses  we  present; 
therefore  institution  type  effects  are  independent  of  affiliation  effects,  and  affiliation  effects  are 
independent  of  the  effects  of  institution  type.  Further,  whenever  we  test  for  significant  differences 
according  to  institution  type  and  departmental  affiliation,  we  test  for  interaction  effects,  and  report 
where  these  are  significant. 

After  the  summary  analysis  of  results  by  institution  type  and  departmental  affiliation,  we 
discuss  a path  model  that  shows  the  relationships  among  many  of  the  variables  in  the  chapter,  and  we 
suggest  how  they  may  work  together  to  foster  high-quality  professional  development.  The  last 
section  of  the  chapter  highlights  the  major  findings  and  suggests  implications  for  both  the  legislation 
and  practice. 


Characteristics  of  IHE/NPO  Recipients  of 
Eisenhower  Grants 


Section  Findings 

♦ A little  less  than  half  of  the  teachers  participating  in  SAHE-grantee  projects  are  in 
projects  in  research  or  doctoral-granting  universities.  A little  more  than  half  are  in 
projects  in  other  types  of  institutions — those  that  grant  bachelor's  and  master's  degrees 
only , private  liberal  arts  institutions , and  public  two-year  colleges. 

♦ Nearly  half  of  the  participating  teachers  are  in  IHEs  whose  Eisenhower  projects  are 
affiliated  with  mathematics  or  science  departments , but  almost  as  many  are  in  IHEs 
whose  Eisenhower  projects  are  affiliated  with  education  departments.  The  majority  of 
participating  teachers  are  in  projects  whose  Eisenhower  projects  are  headed  by  tenured 
professors , and  almost  two-thirds  are  in  projects  that  have  received  Eisenhower  support 
for  four  years  or  more. 

♦ Approximately  one-third  of  the  participating  teachers  are  in  projects  that  provide  one  or 
two  activities , one-third  are  in  projects  that  provide  three  to  ten  activities , and  one-third 
are  in  projects  that  provide  more  than  10  activities.  A little  less  than  half  of  the 
participating  teachers  are  in  projects  that  provide  only  one  or  two  types  of  activities. 

Eighty-six  of  our  total  sample  of  project  directors  are  from  IHEs;  six  are  from  NPOs.  To 
develop  a detailed  view  of  the  characteristics  of  SAHE  grants  in  IHEs,  we  asked  a series  of  questions 
of  each  project  director  about  the  type  of  college  or  university  the  director  works  in,  and  the 
department,  school,  or  center  with  which  the  director  is  affiliated.  We  also  asked  each  director  to 
describe  his  or  her  position  (e.g.,  tenured  professor,  adjunct  professor). 
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As  Exhibit  6.1  shows,  results  from  our  survey  of  SAHE-sponsored  IHE  Eisenhower  project 
directors  indicate  that  about  a third  (34  percent)  of  teachers  participating  in  IHE  Eisenhower-assisted 
activities  are  in  IHEs  that  offer  only  bachelor’s  or  master’s  degrees,  about  two-fifths  (44  percent)  of 
participating  teachers  are  in  IHEs  at  either  research  or  doctoral-granting  universities,  and  less  than  a 
quarter  (22  percent)  are  in  private  liberal  arts  and  public  two-year  universities. 

EXHIBIT  6.1 

Characteristics  of  SAHE  Grantees 


IHEs  (percent  of  teachers  participating 
in  SAHE-sponsored  IHE  projects,  by 
IHE  characteristics)  (n=86) 


NPOs^numbe0jn^6) 


IHE  Institution  Type 

Research  1 6 

Doctoral  28 

Bachelor’s/Master’s-Granting  34 

Private  Liberal  Arts  10 

Public  Two  Year  12 


NPO  Institution  Type 

Professional  Association 
Media  Organization 
Consortium  of  City 
School  Divisions 

Environmental  Organization 
Educational  Institutions 


1 

2 

1 

1 

1 


Number  of  Years  Receiving 


Eisenhower  Support 

Less  than  1 year  1 

1 year  8 

Between  1-3  years  29 

Between  4-5  years  10 

More  than  5 years  48 

Departmental  Affiliation  of 
Principal  Investigator 

Mathematics  23 

Science  24 

Education  39 

Other  13 


1 

0 

3 

2 

0 


Position  of  Principal  Investigator 


Tenured  Professor  56 

Adjunct  Professor  1 

Dean  or  Department  Head  17 

Research  Center  Staff  7 

Other  20 


g 

As  a result  of  rounding,  percents  may  not  sum  to  100  percent. 
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Each  IHE  project  director  also  reported  the  department,  school,  or  center  with  which  he  or 
she  is  affiliated.  As  Exhibit  6. 1 shows,  more  than  a third  (39  percent)  of  teachers  participating  in 
SAHE-sponsored  IHE  projects  are  in  projects  affiliated  with  the  education  school  or  department;  24 
and  23  percent  are  in  projects  affiliated  with  the  science  and  mathematics  department,  respectively; 
and  13  percent  are  in  projects  affiliated  with  other  departments  or  schools.  Examples  of  other 
departments  or  schools  that  project  directors  are  affiliated  with  are  departments  of  broadcasting 
services,  general  studies,  language  and  literature,  and  parallel  studies;  a college  of  business;  a group 
of  academic  administrators;  and  a university  outreach  organization. 

Each  project  director  also  indicated  which  position  he  or  she  held.  Exhibit  6.1  shows  that  the 
majority  of  participating  teachers  (56  percent)  are  in  projects  whose  directors  report  being  tenured 
professors;  17  percent  are  in  projects  whose  directors  are  deans  or  department  heads  (who,  in  most 
cases,  also  is  a tenured  professor);  seven  percent  are  in  projects  whose  directors  are  part  of  the  staff 
of  a research  center;  and  only  one  percent  are  in  projects  whose  directors  report  being  adjunct 
professors.  The  remaining  20  percent  of  participating  teachers  are  in  projects  whose  directors  report 
holding  positions  other  than  one  of  these  four,  but  do  not  describe  the  position. 

We  also  asked  IHE  project  directors  how  long  they  have  been  receiving  Eisenhower  funds, 
not  limited  to  the  project  we  asked  them  about  on  the  survey.  As  Exhibit  6.1  shows,  nearly  half  (48 
percent)  of  the  participating  teachers  are  in  projects  that  have  received  Eisenhower  support  for  more 
than  five  years;  ten  percent  are  in  projects  that  have  received  support  from  the  Eisenhower  program 
for  between  four  and  five  years;  and  29  percent  are  in  projects  that  have  received  Eisenhower  grants 
for  between  one  and  three  years.  Eight  percent  of  participating  teachers  are  in  projects  that  have 
received  Eisenhower  funds  for  less  than  one  year,  and  only  one  percent  are  in  projects  that  have 
received  Eisenhower  support  for  less  than  a year. 

The  NPOs  in  our  sample  represent  several  different  types  of  organizations.  As  indicated  in 
Exhibit  6. 1 , one  NPO  is  a professional  association,  two  are  media  organizations,  one  is  a consortium 
of  several  city  school  divisions,  one  is  a private  nonprofit  environmental  organization,  and  one  is  a 
regional  nonprofit  educational  institution.  Three  of  the  NPOs  have  received  money  from  the 
Eisenhower  program  for  between  one  and  three  years,  two  have  received  Eisenhower  funds  for 
between  four  and  five  years,  and  one  of  the  NPOs  in  our  study  has  received  Eisenhower  funds  for 
less  than  one  year. 

To  get  a sense  of  the  types  and  scope  of  activities  that  SAHE  grantees  provide,  we  asked 
project  directors  to  describe  the  types  of  activities  that  they  provide  (e.g.,  workshops,  conferences, 
courses,  networks,  and  internships),  the  number  of  activities  that  they  provide  each  year,  and  how 
many  teachers  are  served  by  each  activity  (data  not  shown).  Their  responses  indicate  that  SAHE- 
grantees  vary  in  the  number  of  activities  that  they  provide.  Almost  a third  of  participating  teachers 
are  in  SAHE-grantee  projects  that  provide  only  one  or  two  activities  per  year:  specifically,  21  percent 
of  participating  teachers  are  in  projects  that  provide  only  one  activity,  and  10  percent  are  in  projects 
that  provide  two  activities.  Seventeen  percent  of  participating  teachers  are  in  projects  that  provide 
three  to  five  activities,  16  percent  are  in  projects  that  provide  six  to  ten  activities,  16  percent  are  in 
projects  that  provide  1 1 to  15  activities,  and  21  percent  are  in  projects  that  provide  16  or  more 
activities.  Similarly,  some  SAHE  grantees  support  only  a few  types  of  activities  while  others  support 
several  types.  Thirty-nine  percent  of  participating  teachers  are  in  projects  that  support  only  one  type 
of  activity,  eight  percent  are  in  projects  that  support  two  types,  four  percent  are  in  projects  that 
support  three  types,  21  percent  are  in  projects  that  support  four  types,  and  29  percent  are  in  projects 
that  support  five  or  more  types  of  activities. 
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The  median  number  of  teachers  that  SAHE  grantees  serve  is  33  teachers  (data  not  shown).9 
More  specifically,  3 1 percent  of  participating  teachers  are  in  SAHE-grantee  projects  that  serve  25 
teachers  or  fewer,  29  percent  are  in  projects  that  serve  26  to  50  teachers,  13  percent  are  in  projects 
that  serve  51  to  100  teachers,  10  percent  are  in  projects  that  serve  101  to  200  teachers,  nine  percent 
are  in  projects  that  serve  200  to  500  teachers,  and  three  percent  are  in  projects  that  serve  more  than 
500  teachers. 

These  results  indicate  that,  on  average,  professional  development  sponsored  by  SAHEs  are 
projects  in  IHEs;  are  housed  in  education,  mathematics,  or  science  departments;  are  in  institutions 
most  likely  to  grant  teaching  degrees;  directed  by  tenured  professors  who  themselves  or  whose 
institutions  have  a history  of  several  years  of  working  with  the  Eisenhower  program;  and  are  focused 
on  a few  types  of  activities  for  a small  number  of  teachers.  This  contrasts  with  the  professional 
development  provided  by  districts,  as  reported  in  Chapters  4 and  5;  districts  tend  to  provide 
portfolios  of  a diverse  set  of  activities,  while  SAHE  grantees  typically  support  one  focused,  small- 
scale  project.  We  now  move  on  to  discuss  the  structural  characteristics  of  the  professional 
development  activities  that  SAHE  grantees  provide. 


Structural  Features  of  SAHE-Grantee-Provided 
Professional  Development 

Section  Findings 


♦ SAHE  grantees  use  Eisenhower  funds  to  support  activities  mainly  in  mathematics  and 
science , but  more  than  a third  of  teachers  who  participate  in  SAHE-sponsored  projects 
are  in  projects  that  also  use  Eisenhower  money  to  support  activities  in  other  subject 
areas. 

♦ Over  80  percent  of  participating  teachers  are  in  SAHE-sponsored  projects  where  the 
primary  Eisenhower-assisted  activity  is  a traditional  type  (e.g.,  workshops  and  courses); 
only  75  percent  of  participating  teachers  are  in  projects  where  the  primary  activity  is  of  a 
reform  type  (e.g.,  study  groups , teacher  networking  and  mentoring). 

♦ Most  SAHE  grantees'  primary  activities  are  of  long  duration — they  last  more  than  40 
hours  and  span  from  one  month  to  a year;  however,  a small  portion  last  less  than  nine 
hours  and  span  less  than  one  month. 

♦ Most  SAHE  grantees’  primary  activities  projects  do  not  serve  all  teachers  in  a grade, 
department,  or  a whole  school  (i.e.,  collective  participation). 


9 Information  about  the  number  of  teachers  served  by  the  SAHE  grantees  is  an  approximation.  We  had  two  sources 
of  information  about  the  number  of  teachers  served  — data  from  professional  development  activity  lists  that  we 
collected  from  SAHE-grantee  project  directors  and  data  from  our  telephone  surveys  with  SAHE-grantee  project 
directors.  In  many  cases,  these  data  did  not  match  exactly.  We  averaged  the  two  sources  of  data  together  and  used 
this  number  as  our  estimate  of  the  number  of  teachers  served  by  each  SAHE-sponsored  IHE  or  NPO  project. 
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♦ IHE  projects  housed  in  education  departments,  compared  to  mathematics/science 
departments,  have  significantly  more  reform  types  of  activities,  and  have  activities  of 
significantly  longer  duration,  both  in  number  of  contact  hours  and  span  of  time. 
Research/doctoral  universities  sponsor  Eisenhower  professional  development  activities 
with  significantly  more  collective  participation  than  other  types  of  colleges  and 

universities. 

The  substance  of  activities  depend  to  a large  extent  on  the  structure  of  the  activity — 
specifically,  the  type  of  activity,  its  duration,  and  the  groups  of  teachers  who  participate.  The 
legislation  recognizes  the  importance  of  these  features  of  professional  development.  Although  the 
legislation  does  not  specify  the  particular  type  of  professional  development  that  SAHE  grantees 
should  provide,  the  law  draws  on  research  that  says  that  “new  and  innovative  strategies  for  teaching 
to  high  standards  will  require  time  for  teachers,  outside  of  the  time  spent  teaching,  for  instruction, 
practice,  and  collegial  collaboration”  (Section  2001(4)(D)).  There  are  specific  requirements  in  the 
law  that  SAHE-sponsored  IHE/NPOs  provide  professional  development  that  is  “sustained  and 
intensive”  (Section  221 1(b)(1)  and  (2)).  In  addition,  SAHE  grantees  are  expected  to  fulfill  the  more 
general  purposes  of  the  law,  which  call  for  professional  development  that  “is  of  sufficient  intensity 
and  duration  to  have  a positive  and  lasting  impact  on  the  teacher’s  performance  in  the  classroom” 
(Section  2002(2)(E)).  Also,  the  SAHE  portion  of  the  law  says  that  professional  development  may  be 
designed  for  “teams  of  teachers”  (Section  221 1(b)(1)),  among  other  groups.  This  is  consistent  with 
the  part  of  the  law  focused  on  districts,  which  emphasizes  the  importance  of  school-level 
participation,  and  encourages  that  the  professional  development  “take[s]  place  at  the  individual 
school  site”  (Section  2210(a)(1)(B)). 

Although  there  is  limited  research  on  the  relationship  between  features  of  professional 
development  and  teacher  or  student  outcomes,  the  evidence  that  is  available  supports  the  law’s 
emphasis  on  type,  duration,  and  collective  participation.  The  literature  suggests  that  traditional 
methods  of  professional  development,  such  as  workshops,  are  not  likely  to  extend  over  long  periods 
of  time  and  offer  teachers  opportunities  for  in-depth  study  to  practice  what  they  have  learned,  and  to 
collaborate  and  provide  feedback  to  each  other  (Little,  1993).  Also,  since  these  traditional 
approaches  are  less  likely  to  afford  teachers  the  opportunity  for  reflection  on  what  they  have  learned 
and  for  in-depth  engagement,  these  types  of  activities  are  thought  to  be  not  as  likely  to  elicit  the 
desired  changes  in  knowledge,  skills,  and  teaching  practice  (Darling-Hammond,  1997a;  Sparks  & 
Loucks-Horsley,  1989). 

Further,  proponents  of  systemic  reform  maintain  that  teachers  who  teach  the  same  grade  or 
subject  area  should  be  operating  from  the  same  subject  base,  and  from  similar  approaches  to  teaching 
and  learning.  Researchers  suggest  that  professional  development  that  is  designed  for  whole  schools 
or  groups  of  teachers  from  schools  (defined  as  “collective  participation”  in  this  report)  provide 
teachers  with  a community  of  learners,  as  well  as  the  capacity  to  share  knowledge  and  to  learn  from 
each  other,  and  to  develop  and  implement  strategies  to  serve  the  specific  needs  of  their  students 
(Ball,  1996;  Little,  1993;  Newmann  et  al.,  1996). 

In  this  section,  we  report  the  findings  from  our  SAHE-grantee  survey  that  describe  the  three 
structural  features — type,  duration,  and  collective  participation — which  are  emphasized  in  both  the 
Eisenhower  legislation  and  the  professional  development  literature. 
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Subject  Areas  of  Primary  Activity 

We  asked  each  SAHE-grantee  project  director  to  list  the  subject  areas  covered  in  the  project’s 
primary  activity,  including  mathematics,  science  and  other  subject  areas.10  As  Exhibit  6.2  shows,  the 
majority  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose  primary 
Eisenhower-assisted  activity  includes  mathematics  only  (27  percent),  science  only  (18  percent),  or  a 
combination  of  mathematics  and  science  (18  percent).  Twenty-eight  percent  of  participating  teachers 
are  in  projects  whose  primary  activity  includes  a combination  of  mathematics,  science  and  other 
subject  areas,  and  eight  percent  of  teachers  are  in  projects  whose  primary  activity  does  not  include 
mathematics  and  science. 


EXHIBIT  6.2 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  with  Primary  Activities 
That  Cover  Mathematics,  Science,  and  Other  Subject  Areas  (n=92) 


Subject  areas  supported  by  Eisenhower 


Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  27  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
primary  Eisenhower-assisted  activity  is  in  mathematics  only.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating 
teachers  for  each  category. 


Type  of  Activity 

To  examine  the  extent  to  which  SAHE  grantees  provide  traditional  vs.  reform  activities,  we 
asked  SAHE-grantee  project  directors  to  classify  their  primary  activity  using  our  list  of  traditional 
and  reform  types  of  professional  development.  All  types  of  workshops  and  courses  are  categorized  as 

10  Our  sampling  plan  required  that  selected  SAHE-grantee  projects  offer  activities  in  mathematics  and/or  science. 
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traditional,  and  the  seven  other  types  of  activities  are  categorized  as  reform:  study  groups,  teacher 
networking,  mentoring,  committees  or  task  forces,  internships,  individual  research  project,  and 
teacher  resource  centers. 

Results,  shown  in  Exhibit  6.3,  indicate  that  about  three-quarters  (76  percent)  of  participating 
teachers  are  in  projects  that  support  workshops,  and  55  percent  are  in  projects  that  support  workshops 
as  their  primary  activity.  Similarly,  38  percent  are  in  projects  that  offer  college  courses,  and  26 
percent  are  in  projects  that  offer  college  courses  as  their  primary  activity. 

EXHIBIT  6.3 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Offer  “Traditional” 

Activities  (n=92) 
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College  course  Workshops 

Types  of  "traditional"  activities 


□ Offer  activity 

B Offer  activity  as  primary 
focus 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  38  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that  use 
Eisenhower  funds  to  offer  college  courses.  The  second  bar  shows  that  26  percent  of  participating  teachers  are  in  projects  whose  primary 
Eisenhower- assisted  activity  is  a college  course.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for 
each  category. 


Exhibit  6.4  shows  the  percent  of  participating  teachers  in  projects  that  support  each  of  the 
seven  reform  types  of  professional  development.  Between  a third  and  a half  of  participating  teachers 
are  in  projects  that  use  Eisenhower  funds  to  support  teacher  networks  (43  percenjl,  mentoring  (41 
percent),  and  resource  centers  (32  percent).  Twenty-five  percent  are  in  projects  that  use  Eisenhower 
funds  for  study  groups,  and  15  percent  are  in  projects  that  use  Eisenhower  funds  for  committees  or 
task  forces.  Less  than  20  percent  are  in  projects  that  use  money  from  the  Eisenhower  program  to 
fund  either  internships  or  individual  research  projects. 
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These  reform  activities  are  rarely  a SAHE  grantee’s  primary  activity.  Only  nine  percent  of 
participating  teachers  are  in  projects  that  support  teaching  networks  as  their  primary  activity,  four 
percent  are  in  projects  that  support  study  groups  as  their  primary  activity,  and  two  percent  are  in 
projects  that  support  mentoring  as  their  primary  activity.  No  SAHE  grantees  use  Eisenhower  to 
support  committees  or  task  forces,  internships,  individual  research  projects,  or  teacher  resource 
centers  as  their  primary  activity. 


EXHIBIT  6.4 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Offer  “Reform” 

Activities  (n=92) 


force 

Types  of  "reform*  activities 


□ Offer  activity 

■ Offer  activity  as 
primary  focus 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  25  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that  use 
the  Eisenhower  program  to  fund  study  groups.  The  second  bar  shows  that  4 percent  of  participating  teachers  are  in  projects  whose 
primary  activity  is  a study  group.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each  category. 

Among  SAHE  grantees  that  are  IHEs,  we  examined  whether  the  choice  of  supporting  a 
reform  versus  traditional  activity  as  the  primary  activity  differs  by  type  of  institution  or  departmental 
affiliation.11  Exhibit  6.5  shows  that  there  are  no  significant  differences  by  institution  type,  but 
education  departments  and  “other”  departments  are  significantly  more  likely  than 
mathematics/science  departments  to  have  a reform  activity  as  their  primary  activity.  The  difference 
may  be  due  to  the  fact  that  professors  in  education  and  “other”  departments  are  more  likely  to  be 
social  scientists,  or  curriculum  and  instruction  specialists  than  are  mathematics  or  science  professors. 


11  The  distribution  and  mean  for  IHEs  and  NPOs  are  presented  separately  in  these  analyses,  but  we  did  not  test  for 
significant  differences  between  IHEs  and  NPOs  because  of  the  small  number  of  NPOs  in  our  sample. 
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As  a result,  professors  in  education  or  “other”  departments  may  be  more  knowledgeable  about 
alternative  formats  for  structuring  activities,  other  than  traditional  courses  or  workshops. 


EXHIBIT  6.5 

SAHE-grantees’  Support  for  “Reform”  Types  of  Activities,  Overall  and  by  Institution 

Type  and  Departmental  Affiliation  (n=92) 
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Significant  Pairwise  Contrasts 

Departmental  Affiliation  Mathematics/Science  vs.  Education;  Mathematics/Science  vs.  Other 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  Read : The  first  distribution  shows  that  on  average,  14  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
primary  Eisenhower-assisted  activity  is  a reform  activity.  Support  for  reform  activities  differs  significantly  by  departmental  affiliation,  but  not  by 
institution  type.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  IHE/NPO  projects  at  one  data  point  (or  value)  increases,  the  dots 
form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular  category.  The  number  to  the  right 
of  the  distribution  is  the  mean.  


Duration 

In  addition  to  subject-area  focus  and  type  of  activity,  the  duration  of  the  activity  is  another 
important  structural  feature.  Duration  includes  both  contact  hours  and  span  across  time  in  days, 
months,  and  years.  To  measure  the  duration  of  SAHE-grantee  activities,  we  asked  each  project 
director  across  what  period  of  time  the  project’s  primary  activity  extended  and  the  total  number  of 
hours  that  it  lasted. 

Contact  Hours 

Each  SAHE-grantee  project  director  reported  the  number  of  contact  hours  the  typical 
participant  engaged  in  the  grantees’  primary  activity.  Responses,  shown  in  Exhibit  6.6a,  indicate  that 
the  majority  of  participating  teachers  (58  percent)  are  in  projects  that  sponsor  activities  that  last 
more  than  40  hours.  Thirty-two  percent  of  teachers  are  in  projects  whose  primary  activity  lasts 
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between  nine  and  40  hours,  10  percent  are  in  projects  whose  primary  activity  lasts  between  four  and 
eight  hours,  and  only  one  percent  are  in  projects  whose  primary  activity  lasts  less  than  four  hours. 

EXHIBIT  6.6a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  by  Contact 

Hours  of  Primary  Activity  (n=92) 
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Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  one  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose  primary 
Eisenhower-assisted  professional  development  activity  lasts  less  than  four  hours.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of 
participating  teachers  for  each  category. 


We  now  examine  whether  the  number  of  contact  hours  of  activities  differs  significandy  by 
institution  type  or  departmental  affiliation.  Exhibit  6.6b  shows  that  SAHE-sponsored  IHE  projects 
range  in  contact  hours  from  one  to  about  130  hours,  with  an  average  of  64  contact  hours.  Exhibit 
6.6b  also  indicates  that  there  are  significant  interactions  between  institution  type  and  departmental 
affiliation.  The  pattern  of  interactions,  shown  in  Exhibit  6.6c,  indicates  that  SAHE-grantee  projects 
housed  in  education  departments  in  research/doctoral-granting  universities  have  more  than  twice  as 
many  contact  hours  as  other  projects. 
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EXHIBIT  6.6b 


Contact  Hours  of  SAHE-grantees’  Primary  Activity,  Overall  and  by  Institution 
Type  and  Departmental  Affiliation  (n=92) 
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affiliation  is  significant 
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Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  activities  are  in  projects  whose  primary 
activity  averages  64  hours.  The  amount  of  time  of  a project’s  primary  Eisenhower-assisted  professional  development  activity  differs 
significantly  by  departmental  affiliation  but  not  by  institution  type.  Each  dot  represents  one  1HE/NPO  project.  As  the  number  of  IHE/NPO 
projects  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the 
distribution  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 

Note:  Values  were  truncated  at  150  hours,  so  values  above  150  do  not  appear  on  the  distribution. 
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EXHIBIT  6.6c 


Contact  Hours  of  SAHE-grantees’  Primary  Activity,  Interaction  of  Institution  Type  and 

Departmental  Affiliation  (n=86) 


- Research/doctoral 

- Other 


Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  data  point  designated  by  the  first  square  indicates  that  in  mathematics/science  departments  in 
nonresearch/doctoral -gran ting  universities,  the  average  number  of  contact  hours  in  the  primary  Eisenhower-assisted  project  is  58.4.  The  line 
with  data  points  designated  by  squares  indicates  the  number  of  contact  hours  in  projects  of  nonresearch/doctoral-granting  institutions,  in  each 
of  the  three  types  of  departments.  The  line  with  the  data  points  designated  by  diamonds  indicates  the  number  of  contact  hours  in  projects  at 
research/doctoral-granting  institutions  in  each  of  the  three  types  of  departments. 


Span 

To  measure  the  span  of  SAHE-grantee  activities,  we  asked  each  project  director  to  describe 
the  time  period  over  which  the  primary  activity  was  spread,  including  the  main  activity  and  any 
formal  preliminary  or  follow-up  sessions.  We  asked  for  this  information  only  for  primary  activities; 
thus  the  information  is  not  available  for  internships,  teacher  resource  centers,  teacher 
committees/task  forces,  or  individual  research  projects,  which  no  IHE  or  NPO  provided  as  a primary 
activity.  Exhibit  6.7a  shows  that  a little  more  than  a quarter  (27  percent)  of  participating  teachers  are 
in  projects  whose  primary  activity  extends  over  more  than  one  year,  but  the  most  common  span  of 
activities  is  between  one  month  and  one  year;  fifty-two  percent  of  participating  teachers  are  in 
projects  whose  primary  activity  spans  this  period  of  time.  A substantial  portion  of  participating 
teachers  (22  percent)  are  in  projects  that  sponsor  primary  activities  that  span  less  than  one  month. 
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EXHIBIT  6.7a 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  by  Time 
Span  of  Primary  Activity  (n=92) 


month  year 

Span  of  primary  activity 


Source.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  seven  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
primary  Eisenhower-assisted  professional  development  activity  spans  one  day.  Each  bar  and  the  number  on  top  of  it  represent  the  percent 
of  participating  teachers  for  each  category. 


Next  we  examine  whether  span  differs  for  SAHE-supported  IHE  projects  by  institution  type 
or  departmental  affiliation.  The  scale  represented  in  Exhibit  6.7b  has  a range  of  one  to  five,  where 
one  equals  one  day  and  five  equals  greater  than  one  year.  Results  in  Exhibit  6.7b  show  that  projects 
housed  in  mathematics/science  departments  range  in  span  from  one  day  to  one  year,  and  that  no 
projects  in  education  or  “other”  departments  have  a span  of  less  than  one  month.  The  only 
significant  differences  are  that  IHE  projects  in  education  departments  sponsor  activities  that  are 
significantly  longer  in  span  (an  average  of  about  one  month  to  one  year)  than  IHE  projects  in 
mathematics  or  science  departments  (an  average  of  three  days  to  one  month). 


t-'i 

( 


*6-18 


EXHIBIT  6.7b 


Span  of  SAHE-grantees’  Primary  Activity,  Overall  and  by  Institution  Type 
and  Departmental  Affiliation  (n=92) 
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Significant  Pairwise  Contrasts 

Departmental  Affiliation Mathematics/Science  vs.  Education 


Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  projects  are  in  projects  in  which  the  span 
of  the  primary  activity  is  3.8  (where  one=one  day,  two=two  to  seven  days,  three=eight  days  to  one  month,  four=one  month  to  one  year,  and 
five=greater  than  one  year).  The  time  span  of  the  project’s  primary  Eisenhower- assisted  activity  differs  signifi candy  by  departmental 
affiliation  but  not  by  institution  type.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  IHE/NPO  projects  at  one  data  point  (or 
value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular 
category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


In  summary,  our  data  indicate  that  SAHE-grantee  activities  generally  are  of  long  duration:  58 
percent  of  participating  teachers  are  in  projects  whose  primary  activity  has  40  or  more  contact  hours, 
and  79  percent  are  in  projects  whose  primary  activity  spans  more  than  one  month  (see  Exhibits  6.6a 
and  6.7a,  respectively).  Part  of  the  reason  IHE/NPO  activities  have  long  durations  is  that  many  of 
them  are  college  courses  (26  percent  of  participating  teachers  are  in  projects  whose  primary  activity 
is  a college  course),  which,  by  design,  meet  several  hours  per  week  and  are  spread  over  several 
months.  However,  as  we  showed  in  Chapter  3,  even  workshops  and  reform  activities  tend  to  have 
long  durations  when  provided  by  IHE/NPOs.  This  might  be  because  IHEs  adopt  the  paradigm  of 
courses  for  many  of  the  other  types  of  activities  that  they  provide.  Also,  the  proposal  guidelines  that 
SAHEs  establish  might  require  activities  of  long  duration.  Another  possible  explanation  is  that  IHEs 
are  knowledgeable  about  and  apply  principles  of  adult  education,  which  suggest  that  sustained 
activities  are  the  most  useful  and  effective.  This  explanation  is  supported  by  the  fact  that  IHE 
projects  in  education  departments  have  activities  of  longer  duration  than  projects  in  mathematics  or 
science  departments.  We  would  expect  that  project  directors  affiliated  with  education  departments 
would  be  more  familiar  than  project  directors  affiliated  with  mathematics/science  departments  with 
how  to  optimally  design  professional  development  opportunities  for  teachers;  professors  in  education 
departments  at  research  universities  are  the  source  of  most  of  the  current  research  and  theoretical 
literature  on  effective  professional  development. 


Collective  Participation 


The  final  structural  feature  of  professional  development  that  we  discuss  is  collective 
participation,  or  the  extent  to  which  activities  are  geared  toward  the  needs  of  groups  of  teachers  or 
whole  schools,  rather  than  individual  teachers.  To  measure  collective  participation  in  SAHE-grantee 
professional  development  activities,  we  asked  each  project  director  which  of  the  following  groups 
participated  in  the  project’s  primary  activity:  1)  all  teachers  in  department  or  grade-level  groupings, 
and/or  2)  all  teachers  in  a school  or  set  of  schools  (as  opposed  to  teachers  as  individuals  or  teachers 
as  representatives  of  their  department,  grade  level,  or  school). 

Exhibit  6.8a  shows  that  15  percent  of  participating  teachers  are  in  projects  in  which  all 
teachers  in  a department  or  grade  participate  in  the  primary  Eisenhower-assisted  activity,  and  14 
percent  are  in  projects  in  which  that  participation  in  the  primary  activity  includes  participation  from 
all  teachers  in  a school.  These  low  rates  of  collective  participation  may  be  explained  in  part  by  the 
fact  that  IHEs  commonly  provide  college  courses,  and  teachers  usually  enroll  in  courses  individually. 
In  addition,  teachers  often  compete  to  enroll  in  IHE  activities.  Participation  by  groups  of  teachers  or 
whole  schools  in  the  same  activity  would  be  more  likely  to  occur  with  a noncompetitive  application 
process  or  a competition  focused  on  whole  schools,  departments,  or  grades,  rather  than  pre-designed 
workshops  focused  on  individuals  or  courses  for  which  the  IHE/NPO  seeks  applicants. 

EXHIBIT  6.8a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  Whose  Primary  Activity 

Involves  Collective  Participation  (n=92) 


Source.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  14  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
primary  activity  includes  participation  by  all  teachers  in  a school.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of 
participating  teachers  for  each  category. 
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We  averaged  the  two  variables  comprising  the  “collective  participation”  measure  to  analyze 
differences  according  to  institution  type  and  departmental  affiliation.  Analysis  of  this  composite, 
illustrated  in  Exhibit  6.8b,  shows  that  many  SAHE  grantees  have  neither  of  the  two  types  of 
collective  participation  in  their  primary  activity,  while  a few  SAHE  grantees  have  both  types.  Exhibit 
6.8b  also  shows  that  there  are  significant  differences  in  collective  participation  by  institution  type  and 
departmental  affiliation.  Research/doctoral-granting  institutions  are  significantly  more  likely  than 
other  types  of  IHEs  to  have  collective  participation.  Differences  by  departmental  affiliation  are 
significant,  but  post  hoc  tests  show  that  differences  between  any  two  types  of  departments  are  not 
statistically  significant. 


EXHIBIT  6.8b 


Collective  Participation  in  SAHE-grantee  Projects,  Overall  and  by  Institution  Type  and 

Departmental  Affiliation  (n=92) 
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Significant  Pairwise  Contrasts 

Departmental  Affiliation  Overall  significant,  but  planned  comparisons  insignificant 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  projects  are  in  projects  whose  primary 
activity  has  a collective  participation  measure  of  .2,  where  zero=no  collective  participation,  and  one=includes  both  types  of  collective 
participation.  The  extent  of  collective  participation  differs  significantly  by  both  institution  type  and  departmental  affiliation.  Each  dot 
represents  one  IHE/NPO  project.  As  the  number  of  1HE/NPO  projects  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line 
that  increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular  category.  The  number  to  the  right  of  the  distribution 
is  the  mean. 


One  possible  explanation  for  these  results  is  that  “other”  departments  (examples  in  our 
survey  include  broadcasting,  a college  of  business,  and  a university  outreach  department)  may  be 
more  likely  than  mathematics/science  or  education  departments  to  develop  a particular  program  or 
course  designed  specifically  for  their  Eisenhower  project.  In  mathematics,  science,  or  education, 
teachers  may  participate  in  professional  development  activities  that  are  part  of  the  regular  university 
curriculum.  “Other”  departments  may  be  less  likely  than  education  or  mathematics  and  science 


6-21 

BEST  COPY  AVAIU1LI 


260 


departments  to  have  relevant  regular  courses  or  programs  that  would  draw  teachers  from  various 
sources. 

Summary:  Structural  Features  of  SAHE-grantee  Activities 

As  with  districts,  SAHE  grantees  support  predominately  mathematics  and  science  activities, 
but  they  also  support  activities  in  other  subject  areas.  The  reports  from  SAHE-grantee  project 
directors  about  the  structure  of  their  activities  mirror  the  reports  from  teachers,  described  in  Chapter 
3.  SAHE  grantees  generally  offer  traditional  types  of  activities — courses  and  workshops — although 
a few  grantees,  especially  those  in  education  and  “other”  departments,  are  trying  reform  activities. 
SAHE  grantees  have  low  levels  of  collective  participation  in  their  activities,  but  seem  to  structure 
their  primary  activities  to  support  “sustained  and  intensive"  learning.  On  average,  activities  last  over 
60  hours,  and  span  between  one  month  and  a year.  This  provides  a structure  that  could  facilitate  the 
implementation  of  content  knowledge  focus  and  high-quality  learning  strategies.  We  now  turn  to  an 
analysis  of  the  degree  to  which  SAHE-grantee  projects  focus  on  those  dimensions  of  quality. 


Section  Findings 

♦ Most  SAHE  grantees  report  a strong  content  focus  and  the  use  of  many  types  of  active 
learning  opportunities  in  their  primary  activity. 

♦ Projects  in  mathematics/science  departments  have  high  content  focus  in  both 
research/doctoral-granting  and  nonresearch/doctoral-granting  IHEs;  projects  in 
education  and  other  departments  have  high  content  focus  only  in  research/doctoral- 
granting  universities. 

♦ Eisenhower  project  directors  affiliated  with  education  departments  report  significantly 
more  active  learning  opportunities  than  project  directors  affiliated  with 
mathematics/science  departments. 

Activity  type,  duration,  and  the  extent  of  collective  participation  are  the  structural  features  of 
a professional  development  activity.  Our  results  in  Chapter  3 demonstrated  that  these  characteristics 
are  associated  with  core  features — that  is,  the  methods,  curriculum,  and  practices  that  comprise  the 
professional  development  activity.  Key  aspects  of  the  substance  or  core  of  an  activity  are  a focus  on 
content  knowledge  and  opportunities  for  active  learning. 

The  Eisenhower  legislation  does  not  specifically  describe  how  quality  should  be  defined,  but 
it  provides  some  guidance.  The  legislation  states  that  professional  development  provided  under  the 
Eisenhower  program,  whether  by  the  state,  district,  or  an  IHE/NPO,  should  be  a program  that 
“includes  strong  academic  content  and  pedagogical  components”  (Section  2002(2)(C))  and  “reflects 
recent  research  on  teaching  and  learning”  (Section  2002(2)(B)).  Although  the  law  does  not 
specifically  require  activities  to  have  particular  characteristics,  the  legislation  is  founded  on  recent 
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research  in  professional  development  that  describes  attributes  of  high-quality  professional 
development: 


[Professional  development  must  be  focused  on  teaching  and  learning  in  order  to  improve 
the  opportunities  of  all  students  to  achieve  higher  standards  (Section  2001(4)(A)); 
effective  professional  development  focuses  on  discipline-based  knowledge  and  effective 
subject-specific  pedagogical  skills, ...is  interactive  and  collaborative,  motivates  by  its 
intrinsic  content  and  relationship  to  practice,  builds  on  experience  and  learning-by-doing, 
and  becomes  incorporated  into  the  everyday  life  of  the  school  (Section  200 1(4)(B)). 

The  literature  on  professional  development  also  highlights  the  importance  of  content  focus 
and  active  learning  opportunities.  The  limited  research  that  is  available  suggests  that  professional 
development  is  much  more  effective  when  it  focuses  on  the  content  of  subjects  as  well  as  on  how 
students  learn  specific  content,  rather  than  general  non-content-based  teaching  strategies  (Cohen  & 
Hill,  1998;  Fennema  et  al.,  1996;  Kennedy,  1998).  Also,  studies  have  shown  that  professional 
development  that  offers  opportunities  for  active  learning,  such  as  interacting  with  other  participants, 
leading  exercises,  simulating  practice,  and  obtaining  feedback,  are  more  likely  to  foster  increased 
knowledge  and  skills  and  changes  in  teacher  practice  than  other  more  passive  learning  techniques 
(Carey  & Frechting,  1997;  Dari ing-Hammond,  1997b;  Lieberman,  1996;  Schifter,  1996). 

Focus  on  Content  Knowledge 

We  measured  both  the  content  focus  and  active  learning  opportunities  in  each  SAHE 
grantee’s  primary  professional  development  activity.  To  measure  the  extent  to  which  a grantee’s 
primary  activity  is  content-focused,  we  asked  the  project  director  how  much  emphasis  the  primary 
activity  gives  to  deepening  content  knowledge;  responses  include  “no  emphasis,”  “some  emphasis,” 
or  “strong  emphasis.”  Results,  shown  in  Exhibit  6.9a,  indicate  that  five  percent  of  teachers 
participating  in  SAHE-grantee  projects  are  in  projects  whose  primary  activity  places  no  emphasis  on 
content  knowledge,  23  percent  of  participating  teachers  are  in  projects  whose  primary  activity  places 
some  emphasis  on  content  knowledge  in  the  primary  activity,  and  72  percent  are  in  projects  whose 
primary  activity  places  a strong  emphasis  on  content  knowledge. 
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EXHIBIT  6.9a 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  Whose  Primary  Activity 

Focuses  on  Content  Knowledge  (n=92) 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  five  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
primary  activity  has  no  emphasis  on  content  knowledge.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers 
for  each  category. 


Next  we  analyze  content  focus  by  institution  type  and  departmental  affiliation.  Exhibit  6.9b 
shows  that  the  means  on  the  three-point  content  focus  scale  vary  from  one  to  three,  but  most  are 
above  two.  Further,  the  interaction  of  institution  type  and  departmental  affiliation  is  significant. 
Exhibit  6.9c  indicates  that  projects  in  mathematics/science  departments  have  high  content  focus 
regardless  of  the  type  of  IHE,  while  projects  in  education  and  other  departments  have  a high  content 
focus  only  if  they  are  in  research/doctoral  universities. 
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EXHIBIT  6.9b 


Extent  of  Content  Knowledge  Focus  in  SAHE-grantees’  Primary  Activity,  Overall  and 
by  Institution  Type  and  Departmental  Affiliation  (n=92) 


Interaction  of  Institution  and 
Affiliation  is  Significant 


(/> 

3 

O 

0 

d> 

o> 

1 

§ 

c 

+* 

c 

o 

E 

o 

o 


. 2.5 


*2.8  +2.6 


Research  Other 


IHE  NPO  Institution 


*2.7  ,2.6  2.5 


Math/  Education  Other 
Science 

Affiliation 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  projects  are  in  projects  whose  primary  activity 
has  a content  focus  of  2.7,  where  zero=no  emphasis  on  content  and  three=a  strong  emphasis  on  content.  The  interaction  effects  of  institution  type 
and  departmental  affiliation  on  content  knowledge  are  significant.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  IHE/NPO 
projects  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution 
for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


c 

Ma 


6-25 


EXHIBIT  6.9c 


Extent  of  Content  Knowledge  Focus  in  SAHE-grantees’  Primary  Activity,  by 
Institution  Type  and  Departmental  Affiliation  (n=86) 
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Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  data  point  designated  by  the  first  square  indicates  that  the  average  extent  of  content  knowledge  in  the 
primary  SAHE-grantee  project  in  mathematics/science  departments  in  nonresearch  doctoral -granting  universities  is  2.8  (where  zero=no 
emphasis  on  content  and  three=a  strong  emphasis  on  content).  The  line  with  data  points  designated  by  squares  indicates  the  extent  of 
content  emphasis  of  projects  at  nonresearch/doctoral-granting  institutions,  in  each  of  the  three  types  of  departments.  The  line  with  the  data 
points  designated  by  diamonds  indicates  the  extent  of  content  emphasis  of  projects  at  research/doctoral-granting  institutions,  in  each  of  the 
three  types  of  departments. 


These  findings  may  reflect  that  SAHE  grantees  in  research/doctoral-granting  universities  are 
more  likely  than  those  in  nonresearch  universities  to  be  knowledgeable  about  the  importance  of 
content  focus  in  professional  development,  and  more  equipped  to  provide  strategies  to  learn 
substantive  content.  Also,  mathematics/science  departments  may  have  a strong  content  focus  across 
institutions  types  since  those  departments  deal  mainly  with  subject-specific  topics,  not  pedagogical 
techniques. 


Opportunities  for  Active  Learning 

To  measure  the  second  core  feature — opportunities  for  active  learning — we  asked  SAHE- 
grantee  project  directors  about  how  their  primary  activity  helps  participants  use  new  skills  in  their 
classroom.  We  asked  them  the  following  questions: 

♦ Did  participants  receive  coaching  or  mentoring  in  the  classroom? 

♦ Was  participants’  teaching  observed  by  other  participants  and  feedback  provided? 

♦ Did  participants  practice  under  simulated  conditions,  with  feedback? 

♦ Did  you  or  other  activity  leaders  observe  participants’  teaching  and  provide  feedback? 
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♦ Did  participants  meet  formally  with  other  participants  to  discuss  classroom 
implementation? 

♦ Did  participants  communicate  with  you  concerning  classroom  implementation? 

♦ Did  participants  share  their  students’  work  with  you  or  other  participants? 

♦ Did  participants  meet  informally  to  discuss  classroom  implementation? 

♦ Did  participants  develop  curricula  or  lesson  plans  that  you  or  other  participants  reviewed? 

♦ Did  participants  engage  in  the  following  during  the  activity: 

♦ Give  a lecture  or  presentation  of  a lesson  or  unit? 

♦ Conduct  a demonstration  of  a lesson  or  unit? 

♦ Lead  a whole-group  discussion? 

♦ Lead  a small-group  discussion? 

♦ Write  a paper,  report,  or  plan? 

♦ Review  student  work? 

♦ Score  assessments? 

Exhibit  6. 10a  shows  that  more  than  three-fourths  of  participating  teachers  are  in  projects 
whose  directors  report  that  in  their  primary  Eisenhower  activity,  participants  share  work  (83  percent), 
meet  formally  with  other  participants  to  discuss  classroom  implementation  of  the  new  skills  that  they 
learned  in  the  professional  development  activity  (82  percent),  and  communicate  with  the  teacher  (79 
percent).  Most  participating  teachers  are  in  projects  that  offer  activities  that  allow  participants  to 
lead  a small-group  discussion  (71  percent),  conduct  a demonstration  (71  percent),  have  informal 
meetings  (70  percent),  write  a paper  or  report  (68  percent),  give  a lecture  (65  percent),  develop  a 
curriculum  (63  percent),  have  the  leader  observe  their  teaching  and  provide  feedback  (61  percent), 
receive  mentoring  or  coaching  (57  percent),  and  practice  under  simulated  conditions  (57  percent). 

A little  less  than  half  of  participating  teachers  are  in  projects  whose  primary  activity  includes 
opportunities  for  teachers  to  review  student  work  (48  percent),  observe  other  teachers  and  provide 
feedback  (47  percent),  and  lead  whole-group  discussions  (45  percent).  The  least  common  active 
learning  opportunity  is  having  teachers  score  assessments;  only  22  percent  of  participating  teachers 
are  in  projects  that  include  this  method  in  their  primary  professional  development  activity.  So  it 
seems  that  most  SAHE-grantee  primary  activities  use  a number  of  active  learning  strategies;  among 
the  most  common  strategies  are  communication  and  feedback  with  the  activity  leader,  and  among  the 
least  are  strategies  related  to  scoring  and  assessing  student  work. 
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EXHIBIT  6.10a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Provide  Each  of 
Sixteen  Types  of  Opportunities  for  Active  Learning  in  Primary  Activity  (n=92) 


Types  of  active  learning 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  22  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
primary  Eisenhower-assisted  professional  development  activity  uses  the  active  learning  strategy  of  scoring  assessments.  Each  bar  and  the 
number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each  category. 


Combining  the  16  types  of  opportunities  for  active  learning  reported  by  the  SAHE-grantees 
into  a scale  provides  us  with  a measure  of  the  number  of  types  of  opportunities  for  active  learning 
that  SAHE  grantees  offer,  on  a scale  from  1 to  16.  The  index  provides  one  measure  of  the  diversity 
of  learning  strategies.  Exhibit  6. 1 Ob  illustrates  the  variation  in  the  number  of  active  learning 
opportunities  that  grantees  provide  in  their  primary  Eisenhower-assisted  activity.  Some  SAHE- 
grantee  primary  activities  have  as  few  as  two  active  learning  strategies  in  their  primary  activity, 
while  others  have  as  many  as  15.  Exhibit  6.10b  also  indicates  that  the  interaction  of  institute  type 
and  affiliation  is  significant. 
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EXHIBIT  6.10b 


Number  of  Types  of  Opportunities  for  Active  Learning  in  SAHE-grantees’ 
Primary  Activity,  Overall  and  by  Institution  Type  and  Departmental  Affiliation  (n=92) 
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Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  projects  are  in  projects  that  have  an 
average  of  10  active  learning  opportunities  for  teachers  in  their  primary  Eisenhower-assisted  professional  development  activity.  The 
interaction  effects  of  institution  type  and  departmental  affiliation  on  the  number  of  types  of  opportunities  for  active  learning  is 
significant.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  IHE/NPO  projects  at  one  data  point  (or  value)  increases,  the 
dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular  category.  The 
number  to  the  right  of  the  distribution  is  the  mean. 


An  analysis  of  the  pattern  of  interactions,  shown  in  Exhibit  6.10c,  reveals  that  among  projects 
in  education  departments,  those  in  research  universities  allow  more  opportunities  for  active  learning 
than  those  in  nonresearch  universities.  The  opposite  is  true  for  projects  in  mathematics/science 
departments;  they  do  better  in  nonresearch  universities  than  in  research  universities.  Among  projects 
in  research  institutions,  projects  in  the  education  departments  provide  many  more  active  learning 
opportunities  than  projects  in  mathematics/science  departments  (12  compared  to  seven);  but  for 
projects  in  nonresearch  universities,  there  is  not  much  difference  in  the  number  of  active  learning 
opportunities  provided  by  mathematics/science  and  education  departments. 
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EXHIBIT  6.10c 


Number  of  Types  of  Opportunities  for  Active  Learning  in  SAHE-grantees’  Primary 
Activity,  Interaction  of  Institution  Type  and  Departmental  Affiliation  (n=86) 
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Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  data  point  designated  by  the  first  square  indicates  that  in  mathematics/science  departments  in  nonresearch/ 
doctoral-granting  universities,  the  average  number  of  active  learning  opportunities  in  the  primary  SAHE-grantee  project  is  9.8.  The  line 
with  data  points  designated  by  squares  indicates  the  number  of  active  learning  opportunities  in  projects  at  nonresearch/doctoral-granting 
institutions,  in  each  of  the  three  types  of  departments.  The  line  with  the  data  points  designated  by  diamonds  indicates  the  number  of 
active  learning  opportunities  in  projects  at  research/doctoral -granting  institutions,  in  each  of  the  three  types  of  departments. 


Thus,  projects  in  research  universities  do  better  only  if  they  are  affiliated  with  the  education 
department;  otherwise,  nonresearch  universities  have  more  active  learning  opportunities.  This  may 
indicate  that,  unless  they  are  in  an  education  department,  research  university  professors  are  more 
inclined  to  practice  traditional  lecturing  and  non-active  learning  techniques  than  professors  in  other 
types  of  universities.  Education  departments  are  more  equipped  to  offer  alternative  pedagogical 
strategies,  and  are  more  knowledgeable  about  learning  techniques,  perhaps  because  pedagogy  and 
learning  techniques  are  a fundamental  part  of  their  focus  and  curricula.  In  contrast,  mathematics  and 
science  departments  in  research  universities  are  more  likely  to  focus  on  subject  matter  and  less  on 
pedagogical  techniques  such  as  active  learning. 

Further,  projects  in  non-mathematics/science  or  education  departments  at  nonresearch 
universities  do  almost  as  well  in  providing  active  learning  opportunities  as  projects  in  education 
departments  at  research  universities.  These  are  departments  such  as  media  and  broadcasting,  whose 
activities  are  likely  to  focus  on  active,  hands-on  programs. 
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Summary:  Core  Features  of  Professional  Development 


SAHE-grantee  project  directors  generally  report  that  their  primary  Eisenhower-sponsored 
activity  has  a strong  emphasis  on  content  and  offers  many  types  of  active  learning  opportunities. 
These  findings  are  consistent  with  our  data  from  teachers,  reported  in  Chapter  3,  which  indicate  high 
levels  of  content  focus  and  active  learning  opportunities  in  IHE-sponsored  professional  development 
activities.  It  seems  that  IHE/NPOs  structure  their  activities  in  ways  that  facilitate  high-quality  core 
features,  by  establishing  activities  of  long  duration,  and  IHE/NPOs  implement  the  pedagogical 
strategies  that  the  Eisenhower  legislation  and  the  professional  development  literature  emphasize  as 
important  for  changing  teacher  knowledge  and  behavior.  Further,  projects  in  education  and  “other” 
departments  have  a strong  content  focus  only  in  research/doctoral-granting  universities,  while 
projects  in  mathematics/science  departments  have  a high  content  focus  regardless  of  institution  type. 
Projects  in  education  departments  in  research/doctoral-granting  universities  do  better  than  others  in 
active  learning. 

The  question  for  policy  is  to  identify  the  reasons  for  the  high  average  quality  of  SAHE- 
grantee  activities.  One  explanation  is  that  IHE/NPO  projects  are  competitive,  and  the  SAHE  review 
process  may  favor  depth  over  breadth,  and  may  in  some  cases  require  grantees  to  provide  activities  of 
a certain  duration.  Another  explanation  is  that  IHE/NPOs  spend  more  than  twice  as  much 
Eisenhower  money  per  teacher  participation  as  districts  do;  conversely,  IHE/NPOs  may  spend  more 
money  per  teacher  because  the  high-quality  activities  that  they  design  require  it.  In  1997-98,  districts 
spent  an  average  of  $185  per  teacher  participation,  while  IHEs  spent  an  average  of  $5 12  per  teacher 
participation.12  Thus  on  average  IHE/NPOs  distribute  their  funds  across  fewer  teachers  than  districts 
do.  Apparently,  districts  seek  to  reach  as  many  teachers  as  possible  (breadth),  while  IHE/NPOs  seek 
to  provide  a high-quality  professional  development  experience  to  teachers  (depth),  even  though  this 
means  reaching  fewer  teachers.  This  may  reflect  the  different  roles  that  IHE/NPOs  and  districts  play. 
Perhaps  districts  feel  a responsibility  to  serve  all  of  their  teachers,  while  IHE/NPOs  do  not  have  a 
specific  constituency  of  teachers  to  whom  they  feel  responsible. 

Our  analyses  of  differences  by  institution  type  and  departmental  affiliation  can  perhaps  shed 
some  light  on  these  suppositions.  Perhaps  a large  part  of  the  reason  that  projects  in  education 
departments  at  research/doctoral  institutions  support  activities  that  last  longer  and  offer  more 
opportunities  for  active  learning  than  projects  in  mathematics/science  departments  is  that  the 
directors  in  education  departments  are  more  familiar  with  the  research  on  high-quality  professional 
development. 

Finally,  projects  in  mathematics/science  departments  emphasize  content  knowledge 
regardless  of  the  type  of  university  they  are  housed  in,  while  projects  in  education  and  other 
departments  emphasis  content  knowledge  only  if  they  are  in  a research  university.  This  may  suggest 
that  project  directors  at  nonresearch  universities  affiliated  with  education  or  other  (non- 


12  For  1996-97,  states  reported  in  their  SAHE  reports  that  there  were  92,000  teacher  participations  in  professional 
development.  To  calculate  dollars  per  participation,  we  assumed  that  95  percent  of  the  16  percent  of  the  total 
Eisenhower  appropriation  that  is  earmarked  for  IHE/NPOs  goes  to  IHE/NPOs.  The  1997-98  appropriation  was  $310 
million;  thus  dollars  per  participation  for  IHE/NPO  projects  was  $512.  In  the  SEA  reports  in  1996-97,  states 
reported  that  the  number  of  participations  in  professional  development  in  districts  was  equal  to  1.27  million. 
Assuming  that  90  percent  of  the  earmarked  84  percent  of  the  $310  million  Eisenhower  appropriation  goes  to 
districts,  the  dollars  per  participation  for  districts  was  $185.  This  estimate  is  for  Eisenhower  dollars  only;  the  total 
average  cost  for  districts  may  be  one-third  higher  than  estimated  here,  given  Title  IPs  cost  sharing  requirement 
(Section  2209a  and  b). 


mathematics/science)  departments  need  more  training  and  knowledge  about  the  importance  of 
focusing  on  content  and  how  children  learn  specific  content  in  the  structure  and  implementation  of 
their  professional  development  activities. 

IHE/NPO-provided  Eisenhower-assisted  in-service  activities,  described  in  this  chapter 
generally,  have  several  characteristics  of  high  quality.  The  competitive  process  of  the  SAHE 
component  of  the  Eisenhower  program  may  promote  quality;  and  it  may  be  that  only  the  most 
qualified  faculty  members  pursue  and  receive  awards  to  direct  Eisenhower  projects. 


Targeting  and  Recruitment  of  Teachers 


Section  Findings 

♦ Most  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose  directors  say 
that  they  target  Title  I teachers,  and  teachers  in  high-poverty  and  low-achieving  schools. 
The  majority  of  participating  teachers  are  in  projects  whose  directors  report  that  they  do 
not  target  teachers  of  special  education  or  limited-English  proficient  students. 

♦ Over  three-fourths  of  participants  in  SAHE-grantee  projects  volunteer  to  participate,  and 
most  of  the  rest  are  selected  by  the  principal. 

♦ Most  participating  teachers  are  in  projects  that  try  to  increase  participation  in  their 
professional  development  activities  by  publicizing  them,  and  about  half  are  in  projects 
that  try  to  increase  participation  by  tailoring  the  focus  of  the  activities  and/or  using 

incentives. 


Teachers  cannot  benefit  from  high-quality  professional  development  activities  if  they  do  not 
participate  in  them.  While  individual  initiative  influences  the  extent  to  which  teachers  take 
advantage  of  opportunities,  teachers’  opportunities  for  professional  development  also  are  shaped,  in 
part,  by  the  extent  to  which  they  are  targeted  and  recruited  to  participate  in  activities.  As  we 
discussed  in  Chapters  3 and  4,  the  Eisenhower  legislation  emphasizes  the  importance  of  addressing 
the  needs  of  teachers  of  diverse  student  populations.  These  general  provisions  of  the  law  for  teachers 
of  students  of  diverse  needs  apply  not  only  to  districts,  but  also  to  SAHE  grantees.  Specifically,  the 
law  provides  that  teachers  (and  others)  should  have  access  to  professional  development  that 
incorporates  effective  strategies,  techniques,  methods,  and  practices  for  meeting  the  educational 
needs  of  diverse  student  populations,  including  females,  minorities,  individuals  with  disabilities, 
limited-English-proficient  individuals,  and  economically  disadvantaged  individuals,  in  order  to 
ensure  that  all  students  have  the  opportunity  to  achieve  challenging  state  student  performance 
standards  (Section  2002(2)(D)). 


6-32 


291 


Increasing  the  participation  of  teachers  of  diverse  student  populations  is  important  because 
teachers  in  schools  with  high  populations  of  at-risk  students  are  generally  less  experienced,  have 
fewer  resources  in  their  schools,  and  face  students  who  are  often  more  challenging  to  teach  (Darling- 
Hammond,  1997a;  U.S.  Department  of  Education,  1999a).  These  teachers  are  often  more  in  need  of 
professional  development  than  their  counterparts  in  middle-class  schools  (Darling-Hammond,  1997a; 
U.S.  Department  of  Education,  1999a),  and  they  also  are  less  likely  to  participate  in  professional 
development  activities  (U.S.  Department  of  Education,  1998a). 

Targeting  Teachers  of  Special  Populations  of  Students 

To  see  how  well  SAHE  grantees  target  their  professional  development  practices  to  teachers 
of  the  diverse  student  populations  outlined  in  the  legislation,  we  asked  SAHE-grantee  project 
directors  a series  of  questions  about  their  targeting  and  recruitment  practices.13  We  asked  them  how 
much  emphasis  they  give  to  recruiting  different  types  of  teachers  for  their  activities — Title  I teachers, 
special  education  teachers,  teachers  of  limited-English-proficient  students,  teachers  from  schools 
with  low-achievement  levels,  and  teachers  from  high-poverty  schools  (i.e.,  schools  in  which  50 
percent  or  more  students  are  eligible  for  free  or  reduced-price  lunch).  Exhibit  6.1 1 shows  the  percent 
of  teachers  in  SAHE-grantee  projects  whose  project  director  reports  placing  “no  particular 
emphasis,”  “some  emphasis,”  or  a “strong  emphasis”  on  each  of  these  groups  of  teachers. 

About  three-quarters  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that 
report  placing  some  or  a strong  emphasis  on  recruiting  teachers  in  high-poverty  schools  (72  percent) 
and  low  achieving  schools  (71  percent).  A little  more  than  half  of  participating  teachers  are  in 
projects  that  emphasize  recruiting  Title  I teachers  (55  percent),  and  more  than  one-quarter  are  in 
projects  that  target  special  education  teachers  (31  percent)  and  teachers  of  limited  English  proficient 
students  (29  percent). 

To  measure  the  extent  of  targeting,  we  created  a scale  that  combines  the  emphasis  SAHE 
grantees  give  to  recruiting  the  five  types  of  teachers  shown  in  Exhibit  6.1 1,  where  l=no  particular 
emphasis,  2=some  emphasis,  and  3=strong  emphasis.  An  analysis  of  the  scale  indicates  that  the 
extent  of  targeting  does  not  differ  significantly  by  institution  type  or  departmental  affiliation. 

One  reason  that  SAHE  grantees  report  high  levels  of  targeting  for  teachers  of  some  groups  of 
students  may  be  due  to  SAHE  requirements  for  targeting  special  populations  of  teachers.  In  one  of 
our  case  studies  in  Kentucky,  the  SAHE  Eisenhower  coordinator  indicates  that  Eisenhower  proposals 
from  IHEs  must  ensure  opportunity  for  equitable  participation  by  teachers  of  historically 
underrepresented/underserved  segments  of  society.  This  consideration  extends  to  gender,  economics, 
disabilities,  and  racial  minorities.  In  addition,  the  SAHE  reports  making  aggressive  efforts  to  recruit 
for  participation  teachers  who  are  racial  minorities,  and  teachers  of  minority  or  disabled  students. 

For  example,  the  SAHE  supports  specific  projects  designed  for  particular  groups  of  teachers.  One 
project  is  designed  to  introduce  methodology,  technology,  and  classroom  practices  for  teachers  of 
special-need  students,  and  another  project  is  designed  for  racial  minority  students  to  be  involved  and 
participate  in  particular  exercises  in  the  professional  development  activities.  Similarly,  in  Texas,  an 
IHE  project  coordinator  reported  that  although  teachers  in  Title  I schools  are  not  specifically  targeted 
by  districts  for  professional  development,  SAHE  professional  development  grants  have  always 


13  Data  presented  in  the  next  three  sections,  on  targeting,  alignment  and  coordination,  and  continuous  improvement, 
refer  to  general  SAHE-grantee  activities,  not  just  the  primary  activity. 


EXHIBIT  6.11 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  Whose  Directors  Report 
Placing  No,  Some,  or  a Strong  Emphasis  on  Recruiting  Teachers  of  Special  Student 

Populations  (n=92) 
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Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  45  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
director  reports  placing  no  emphasis  on  recruiting  Title  I teachers;  37  percent  of  participating  teachers  are  in  projects  whose  director  reports 
placing  “some”  emphasis  on  recruiting  Title  I teachers;  and  18  percent  of  participating  teachers  are  in  projects  whose  director  reports  placing 
a “strong”  emphasis  on  recruiting  Title  I teachers.  Each  shaded  section  of  the  bar  and  the  number  at  the  top  of  it  represent  the  average 
percent  of  participating  teachers  for  each  category. 


targeted  teachers  of  underrepresented  minority  and  disadvantaged  students  for  recruitment  and 
participation. 

Our  survey  data  show  that  less  than  20  percent  of  teachers  participating  in  SAHE-grantee 
projects  are  in  projects  whose  director  places  a “strong”  emphasis  on  targeting  teachers  in  Title  I 
schools.  In  our  case-study  interviews,  the  project  director  at  a university  in  Texas  offers  an 
explanation  for  why  she  does  not  target  Title  I teachers.  She  says  that  there  is  no  specific  targeting  of 
Title  I teachers  because  so  many  of  the  schools  that  they  work  with  are  Title  I schools;  only  two  or 
three  of  the  schools  in  the  three  districts  that  she  works  with  are  not  Title  I schools. 

In  addition  to  targeting,  we  also  examined  how  teachers  come  to  participate  in  SAHE-grantee 
professional  development  activities.  We  asked  SAHE-grantee  project  directors  what  percent  of  the 
teachers  in  their  Eisenhower  activities  come  to  participate  in  each  of  several  ways:  volunteering, 
selection  by  their  principal  or  other  administrator,  rotation,  or  other  ways.  Exhibit  6.12  shows  the 
most  common  ways  teachers  come  to  participate  in  SAHE-grantee  projects. 
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EXHIBIT  6.12 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  According  to  How 

Teachers  Come  to  Participate  (n=92) 
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Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998.  v 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  78  percent  of  participating  teachers  are  in  SAHE-grantee  projects  in 
which  participation  comes  from  volunteers.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  IHE/NPO  projects  at  one  data 
point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for  that 
particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


By  far  the  most  common  method  is  having  teachers  volunteer;  seventy-eight  percent  of 
participations  in  SAHE-grantee  projects  come  about  by  having  participants  volunteer.  Twenty 
percent  of  participations  occur  through  principal  selection.  The  remaining  channels  of  participation 
comprise  less  than  five  percent  of  total  participations.  SAHE  grantees  vary  in  the  extent  to  which 
they  rely  on  these  methods.  For  example,  some  participating  teachers  are  in  projects  that  rely 
exclusively  on  teacher  volunteers,  while  some  are  in  projects  that  rely  entirely  on  teachers  selected 
by  principals. 

Methods  of  Increasing  Teacher  Participation 

We  asked  SAHE  grantees  how  they  try  to  increase  the  participation  of  teachers, 
paraprofessionals,  or  others  in  their  activities.  Choices  included  publicizing  activities,  using 
incentives,  tailoring  the  focus  of  professional  development  toward  the  needs  of  special  populations, 
or  other  strategies.  Exhibit  6.13  shows  which  of  these  strategies  SAHE  grantees  use.  The  method 
used  by  most  SAHE-grantee  project  directors  is  publicizing  activities  (83  percent  of  participating 
teachers  are  in  projects  that  use  this  method).  About  half  of  participating  teachers  are  in  projects  that 
use  incentives  to  increase  participation  (51  percent)  and  tailor  the  focus  of  their  activities  (50 
percent);  only  18  percent  are  in  projects  that  use  other  methods  for  increasing  participation. 


EXHIBIT  6.13 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Use  Various 
Strategies  to  Increase  Participation  (n=92) 
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Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

to  read  this  exhibit : The  first  bar  shows  that  83  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that 
publicize  activities  to  increase  participation.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each 
category. 


Summary:  Targeting  and  Recruitment  of  Teachers 

Despite  IHE/NPO  efforts  at  recruiting,  the  effect  that  this  has  on  actual  participation  is 
unclear,  since  nearly  80  percent  of  participants  are  volunteers.  Teachers  of  disadvantaged  students 
may  not  participate  in  high  numbers  even  though  the  activity  may  be  targeted  toward  them.  In 
Chapter  3 we  reported  that  the  actual  participation  rates  of  these  teachers  in  Eisenhower-assisted 
activities  are  much  lower  than  the  SAHE-grantee  reports  of  targeting  would  predict.  The  heavy 
reliance  on  volunteer  participants  might  help  to  explain  why  these  participation  rates  do  not  reflect 
reported  targeting  efforts.  Alternative  targeting  and  recruitment  efforts,  such  as  sponsoring  activities 
in  which  the  whole  school  participates,  may  be  effective  in  increasing  the  participation  of  teachers  of 
special  populations  of  students,  but  in  some  cases  this  approach  may  be  unrealistic  to  implement  on  a 
large  scale. 
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Building  a Vision  for  Professional  Development: 
Alignment  with  Standards  and  Assessments,  and 
Coordination  with  Other  Programs 


Section  Findings 

♦ Most  teachers  in  SAHE-grantee  projects  are  in  projects  whose  directors  say  that  state 
standards  play  a role  in  the  design  of  their  Eisenhower  professional  development; 
however,  state  assessments  and  district  standards  and  assessments  are  much  less  likely  to 
play  a role  in  project  design.  Projects  in  nonresearch  universities  are  better  aligned  with 
state  and  district  standards  and  assessments  than  projects  in  research  universities. 

♦ Few  participating  teachers  are  in  projects  that  have  ongoing  feedback  mechanisms  with 
districts,  work  with  the  district  Eisenhower  coordinators,  or  co-fund  with  federal 
programs.  However,  most  participating  teachers  are  in  projects  that  work  with  districts 
in  other  ways.  These  include  working  closely  with  staff  from  other  federal  programs, 
communicating  with  district  staff,  and  relying  on  district  needs  assessments  to  plan  the 
professional  development  project. 

♦ IHE  projects  housed  in  education  departments  engage  in  significantly  more  types  of 
coordination  with  districts  than  do  projects  in  mathematics/science  departments. 

State  and  district  standards  and  assessments  provide  a vehicle  for  unifying  reforms  and 
professional  development.  Thus  one  method  of  designing  and  developing  a program  of  professional 
development  is  to  base  the  activities,  pedagogy  and  curriculum  on  standards  or  assessments  adopted 
by  the  state  or  district,  and  to  work  with  other  programs  in  the  state  and  district  to  develop  a coherent 
reform  strategy 

In  the  law,  Congress  intended  that  SAHE  grantees  be  part  of  system-wide  reform  efforts. 

The  law  requires  SEAs  to  develop  a state  professional  development  plan  to  improve  teaching  and 
learning  and  to  develop  the  plan  in  conjunction  with  SAHEs  (Section  2205  (b)(2)(A)).  Similarly, 
SAHE  grantees  must  follow  the  law’s  general  provisions  for  alignment  with  state  standards. 
Specifically,  SAHE-sponsored  professional  development  should  help  to  ensure  that  professional 
development  is  linked  to  state  content  and  performance  standards  (Section  2002(2)(A)). 

In  addition  to  general  provisions  about  alignment,  the  Eisenhower  program  specifically 
describes  particular  coordination  requirements  for  IHE/NPO  projects.  While  the  SAHE  component 
of  the  program  operates  separately  from  the  LEA  component  of  the  program,  IHE/NPOs  that  receive 
Eisenhower  grants  are  expected  to  work  in  conjunction  with  LEAs.  Specifically,  the  legislation 
provides  that 

...the  State  agency  for  higher  education,  working  in  conjunction  with  the  State  educational 

agency  (if  such  agencies  are  separate),  shall  make  grants  to,  or  enter  into  contracts  or 

cooperative  agreements  with,  institutions  of  higher  education  and  nonprofit  organizations  of 

demonstrated  effectiveness,  including  museumsand  educational  partnership  organizations, 
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which  must  work  in  conjunction  with  a local  educational  agency,  consortium  of  local 
educational  agencies,  or  schools... (Section  221 1(a)(1)). 

In  fact,  the  Eisenhower  legislation  requires  that  SAHE  grantees  establish  a formal 
relationship  with  one  or  more  school  districts.  It  states  that 

No  institution  of  higher  education  may  receive  assistance  under  (a)(1)  of  this  subsection 
unless  the  institution  enters  into  an  agreement  with  a local  educational  agency,  or  consortium 
of  such  agencies,  to  provide  sustained,  high-quality  professional  development  for  the 
elementary  and  secondary  school  teachers  in  the  schools  of  each  such  agency  (Section 
2211(a)(3)). 

The  importance  of  alignment  and  coordination  also  is  emphasized  in  the  literature.  Linking 
professional  development  with  other  programs  and  reforms  helps  to  provide  a coherent  vision  of 
professional  development  (Elmore  & Burney,  1996;  Guskey,  1997),  and  teachers  recognize  when 
there  are  inconsistencies  among  these  factors  (Grant,  Peterson,  & Shojgreen-Downer,  1996). 
Researchers  have  emphasized  the  important  role  that  standards  and  assessments  can  play  in  designing 
and  implementing  professional  development  and  how  coordination,  in  the  form  of  co-funding,  can 
bring  coherence  to  a professional  development  plan  (Elmore  & Burney,  1996).  While  this  literature 
focuses  on  school  districts,  SAHE  grantees  also  can  be  a part  of  the  overall  vision  if  they  are  aligned 
and  coordinated  with  the  districts. 

Although  SAHEs  are  not  in  the  same  position  as  districts  are  to  build  professional 
development  as  part  of  a systemic  reform  strategy,  they  can  establish  their  own  vision  for  integrating 
their  grantee  activities  into  district  and  state  reform  strategies,  and  in  this  way  can  be  part  of  the 
district  and  state’s  vision  for  reform.  The  extent  to  which  SAHE-grantee  projects  link  their 
professional  development  design,  implementation  and  management  with  district  and  state  reforms 
can  be  measured  by  the  projects’  1)  alignment  with  state  and  district  standards  and  assessments,  and 
2)  extent  of  coordination  with  the  district  and  with  other  federal  programs. 

Alignment  of  Eisenhower-assisted  Activities  with  State  and  District  Standards 
and  Assessments 

To  test  the  extent  to  which  SAHE-grantee  activities  are  aligned  with  state  and  district 
standards  and  assessments,  we  asked  SAHE-grantee  project  directors  if  “state  standards  or 
frameworks”  and  “state  assessments”  played  a role  in  designing  their  Eisenhower  project;  we  asked 
similar  questions  about  district-level  standards  and  assessments.  Responses,  shown  in  Exhibit  6. 14a, 
indicate  that  standards  are  more  likely  to  play  a role  in  project  design  than  assessments.  Almost  all 
teachers  participating  in  SAHE-grantee  projects  (92  percent)  are  in  projects  that  report  that  state 
standards  play  a role  in  project  design,  while  only  62  percent  of  participating  teachers  are  in  projects 
in  which  district  standards  play  a role  in  project  design.  Fifty-seven  percent  are  in  projects  in  which 
state  assessments  play  a role  in  project  design,  compared  to  40  percent  in  projects  in  which  district 
assessments  play  a role. 
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EXHIBIT  6.14a 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  in  Which  State  and 
District  Standards  and  Assessments  Play  a Role  in  Project  Design  (n=92) 


O) 

fs 

.S-  o 

0 0) 

1 2 

Q. 


(0 

Q. 

(0 


Q) 
2 $> 

* c 

o 2 

<Q  D) 

m i 

+5  LU 

2< 
c « 
a>  c 
o •- 

h- 

0) 

CL 


100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10%’ 

0%- 


92% 


57% 


State 


□ Standards  Play  a Role 

■Assessments  Play  a 
Role 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit*.  The  first  bar  shows  that  92  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
project  directors  report  that  state  standards  play  a role  in  their  Eisenhower  project  design.  Each  bar  and  the  number  on  top  of  it 
represent  the  percent  of  participating  teachers  for  each  category. 


For  SAHE  grantees,  as  with  districts,  standards  are  much  more  likely  to  play  a role  in 
Eisenhower  activities  than  are  assessments.  This  may  be  because  more  states  have  standards  than 
assessments,  and  even  states  with  assessments  may  not  have  their  assessments  well-aligned  with  their 
standards.  Further,  state  standards  are  more  likely  than  district  standards  to  play  a role  in  SAHE- 
grantee  project  design.  Perhaps  because  SAHE  grantees  submit  their  proposals  to  the  state,  not  to 
districts,  there  may  be  an  automatic  feedback  and  accountability  loop  that  encourages  grantees  to  be 
responsive  to  state  standards.  Several  IHE  project  directors  offer  evidence  of  the  importance  of  this 
link  with  the  state.  One  project  director  from  Ohio  explains  that  her  state  has  outcome  requirements 
that  are  mandated,  and  the  Eisenhower  project  is  adapted  to  meet  those  state  outcomes  and  meet  the 
standards  set  by  the  state.  A New  York  IHE  project  director  also  says  that  the  implementation  of  the 
Eisenhower  program  was  a response  to  statewide  reforms  in  the  mathematics  and  science  curriculum. 
Similarly,  an  IHE  project  director  in  Kentucky  says  that  the  Eisenhower  program  in  her  state  gives 
preference  to  projects  that  relate  to  the  state’s  learning  goals  and  academic  expectations  for 
mathematics  and  science.  Preference  is  given  to  projects  aligned  with  the  state  assessments’  core 
content,  the  Kentucky  Instructional  Results  Information  System  (KIRIS). 

Further,  district  standards  may  be  more  difficult  to  incorporate  into  SAHE-grantee 
professional  development  activities  because  grantees  often  work  with  multiple  districts.  Also,  in  the 
proposal  that  IHE/NPOs  submit  to  the  SAHE,  they  have  to  detail  the  Eisenhower  professional 


development  activities  that  they  have  planned.  In  some  cases,  identification  of  a district  may  occur 
after  the  grant  is  awarded,  so  district  standards  would  have  little  or  no  influence  on  the  design  and 
development  of  the  activity.  If  this  is  the  case,  requiring  IHE/NPOs,  in  the  proposal  process,  to 
identify  and  connect  with  the  district  from  which  they  will  draw  participants  for  their  Eisenhower 
project  would  help  foster  responsiveness  to  district  goals  and  standards.  As  it  now  stands,  it  seems 
that  the  requirement  for  IHE/NPOs  to  “enter[s]  into  an  agreement”  with  the  LEA  with  which  they 
plan  to  work  may  not  be  as  visible  in  some  state  competitions  as  it  could  be.  It  could  also  be  the  case 
that  some  existing  agreements  have  more  form  than  substance.  Our  measures  of  alignment  are 
limited,  however,  because  they  do  not  capture  the  extent  to  which  informal,  on-going  communication 
and  collaboration  between  IHE/NPOs  and  districts  might  affect  alignment. 

To  examine  whether  the  extent  of  alignment  varies  by  institution  type  and  departmental 
affiliation,  we  created  a composite  measure  of  alignment.  We  added  the  responses  to  the  four 
alignment  questions;  the  scale  ranges  from  0 to  4,  where  0 means  no  state  or  district  standards  or 
assessments  play  a role  in  project  design,  and  4 means  standards  and  assessments  from  both  the  state 
and  districts  play  a role  in  project  design.  Results,  shown  in  Exhibit  6.14b,  indicate  that  there  is 
variation  across  IHE/NPOs  in  the  extent  to  which  projects  are  aligned. 

EXHIBIT  6.14b 

Degree  of  Alignment  between  SAHE-grantees’  Eisenhower  Project  and  State  and 
District  Standards  and  Assessments,  Overall  and  By  Institution  Type  and 

Departmental  Affiliation  (n=92) 
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Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  projects  are  in  projects  that  have  an  average 
alignment  of  2.5,  on  a scale  of  zero  to  four,  where  zero  indicates  no  alignment  and  four  indicates  alignment  with  district  and  state  standards 
and  assessments.  Alignment  differs  significantly  by  departmental  affiliation  but  not  by  institution  type.  Each  dot  represents  one  1HE/NPO 
project.  As  the  number  of  IHE/NPO  projects  at  one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length. 
Each  distribution  represents  the  distribution  for  that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean, 
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Projects  do  not  differ  significantly  in  alignment  according  to  departmental  affiliation,  but 
nonresearch/doctoral-granting  universities  report  more  alignment  with  state  and  district  standards  and 
assessments  than  do  research/doctoral-granting  universities.  This  finding  may  reflect  the  fact  that 
“other”  colleges  are  more  likely  than  research  universities  to  have  large  teacher  training  programs, 
and  thus  are  more  likely  to  work  with  states  and  districts.  Also,  it  may  be  that  project  directors  at 
research/doctoral-granting  universities  are  more  likely  to  have  their  own  research  agendas  that  they 
use  to  shape  their  Eisenhower  projects.  In  contrast,  project  directors  at  nonresearch/doctoral-granting 
universities  are  less  likely  to  be  engaged  in  ongoing  research,  and  thus  may  be  more  inclined  to  shape 
their  project  around  the  state’s  and  district’s  goals  and  interests. 

Coordination  with  Other  Programs 

To  be  a part  of  building  an  effective  professional  development  strategy,  SAHE-grantee 
projects  need  to  be  coordinated  with  other  programs  in  the  district  in  which  they  are  working.  We 
asked  several  questions  to  measure  the  extent  to  which  SAHE  grantees  coordinate  their  efforts  with 
local  districts. 

The  legislation  requires  each  SAHE  grantee  to  establish  a cooperative  agreement  with  the 
LEA  and/or  districts  with  which  it  works  (Section  221 1(3)).  Of  those  teachers  participating  in 
SAHE-grantee  projects  that  work  with  only  one  district,  58  percent  are  in  projects  whose  directors 
have  formal  cooperative  agreements  with  the  district.  Of  those  in  projects  that  work  with  multiple 
districts,  71  percent  are  in  projects  whose  directors  report  having  formal  agreements  with  one  or 
more  of  these  districts.  Since  these  data  are  project  director  self-reports,  the  lack  of  formal 
agreements  is  probably  at  least  as  great  as  the  data  show,  and  these  results  suggest  that  many  SAHE 
grantees  are  not  complying  with  the  legislation’s  directive  to  secure  an  agreement  with  the  district. 

In  addition  to  asking  about  formal  agreements,  we  asked  a series  of  questions  about  how 
SAHE  grantees  interact  with  district  staff.  We  identified  five  separate  dimensions  of  coordination:  1) 
feedback  mechanisms;  2)  support  and  extension  of  district  activities;  3)  ways  of  working  with 
districts;  4)  district  involvement  in  planning,  implementing,  or  monitoring;  and  5)  co-funding  with 
other  federal  programs. 

Feedback  Mechanisms 

To  examine  the  required  feedback  mechanisms  between  SAHE  grantees  and  districts,  we 
asked  each  project  director  if  either  of  the  following  feedback  mechanisms  takes  place  between  the 
project  and  the  district:  regular  required  reports  and  required  evaluations  of  professional  development 
activities.  Exhibit  6.15  shows  that  18  percent  of  participating  teachers  are  in  projects  that  are 
required  to  provide  districts  with  an  evaluation  of  professional  development  activities,  and  14  percent 
are  in  projects  that  are  required  to  complete  reports  for  the  district.  We  conclude  that  IHE/NPO 
projects  generally  do  not  feel  responsible  to  districts;  however,  they  do  demonstrate  a responsibility 
toward  states,  possibly  because,  as  previously  discussed,  it  is  through  states  that  they  receive  their 
Eisenhower  funds. 


EXHIBIT  6.15 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  have  Ongoing 
Feedback  Mechanisms  with  Districts  (n=92) 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit.  The  first  bar  shows  that  14  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects 
whose  directors  say  that  regularly  required  reports  exist  as  an  ongoing  feedback  mechanism  between  them  and  the  district.  Each  bar 
and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each  category. 


Support  and  Extension  of  District  Activities 

Another  dimension  of  coordination  is  the  extent  to  which  SAHE-grantee  projects  are 
designed  to  support,  extend,  and  coordinate  with  district  activities.  We  asked  each  SAHE-grantee 
project  director  if  the  Eisenhower  project  was  designed  to  support  or  extend  professional 
development  activities  in  the  district,  and  whether  the  Eisenhower  project  was  coordinated  with 
district  programs  or  reforms.  Responses,  shown  in  Exhibit  6.16,  show  that  almost  three-quarters  (74 
percent)  of  participating  teachers  are  in  projects  whose  directors  report  that  their  activities  are 
designed  to  support  or  extend  district  professional  development  activities.  However,  less  than  one- 
quarter  (24  percent)  of  participating  teachers  are  in  projects  whose  directors  report  coordinating  with 
district  programs  and  reforms.  These  results,  again,  probably  reflect  the  SAHE-grantee  project 
application  process.  Districts  and  IHE/NPOs  may  choose  to  work  with  each  other  when  the 
IHE/NPO  project  fits  with  district  professional  development  activities,  but  IHE/NPO  projects  are  less 
likely  to  be  coordinated  with  district  reforms  and  programs  than  with  state  standards,  because 
IHE/NPOs  must  be  responsive  to  state  priorities  in  their  grant  proposals. 
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EXHIBIT  6.16 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Support  and 
Coordinate  with  District  Professional  Development  Activities  and  Programs  (n=92) 


Type  of  SAHE-grantee  coordination  with  districts 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit'.  The  first  bar  shows  that  74  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  designed  to 
support  or  extend  other  district  professional  development  activities.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of 
participating  teachers  for  each  category. 

Working  with  Districts 

We  asked  each  SAHE-grantee  project  director  if  he  or  she  worked  with  districts  in  any  of  the 
following  ways:  1)  participate  in  district  planning,  2)  communicate  periodically  with  staff,  and  3) 
rely  on  district  needs  assessments  to  plan  their  project.  We  also  asked  if  districts  conducted  regular 
visits  and  observations  of  professional  development  activities  provided  by  the  SAHE  grantees  and 
whether  the  grantee  worked  with  districts  in  other  ways. 

As  indicated  in  Exhibit  6.17,  many  of  the  participating  teachers  are  in  projects  whose 
directors  report  working  with  districts  in  each  of  these  five  ways.  Ninety  percent  of  participating 
teachers  are  in  projects  whose  directors  communicate  periodically  with  district  staff,  7 1 percent  are  in 
projects  that  rely  on  district  needs  assessments,  53  percent  are  in  projects  that  participate  in  district 
planning,  38  percent  are  in  projects  whose  directors  report  that  districts  visit  and  observe  regularly, 
and  64  percent  are  in  projects  that  work  with  districts  in  other  ways. 
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EXHIBIT  6.17 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Work  with  Districts 

in  Different  Ways  (n=92) 


Participate  in  Communicate  Use  District  Regular  District  Other  Ways 
District  with  Staff  Needs  Visits 

Planning  Assessment 


Ways  SAHE-grantees  work  with  districts 


Source-.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  53  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that  work 
with  their  district  by  participating  in  district  planning.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating 
teachers  for  each  category. 


District  Involvement  in  Planning,  Implementing,  or  Monitoring 

To  measure  the  fourth  component  of  coordination — involvement  of  district  staff — we  asked 
each  project  director  if  the  district  Eisenhower  coordinator(s)  and/or  other  district  administrators  are 
involved  in  planning,  implementing,  or  monitoring  the  SAHE-grantee  project.  Responses,  illustrated 
in  Exhibit  6.18,  indicate  that  53  percent  of  participating  teachers  are  in  projects  whose  directors 
report  working  with  non-Eisenhower  district  administrators  in  planning,  implementing  or  monitoring 
the  IHE/NPOs’  Eisenhower  project.  Surprisingly,  only  24  percent  of  participating  teachers  are  in 
projects  whose  directors  say  they  work  with  the  district  Eisenhower  coordinator  in  these  ways. 
However,  more  than  half  of  participating  teachers  are  in  projects  whose  directors  work  with  other 
district  administrators.  Working  with  district  staff  would  seem  to  afford  SAHE-grantee  project 
directors  the  opportunity  to  become  knowledgeable  about  and  integrate  district  programs,  standards, 
and  assessments  into  the  project;  nevertheless,  many  do  not  seem  to  be  doing  this. 
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EXHIBIT  6.18 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  in  Which  District  Staff 
Are  Involved  in  Planning,  Implementing,  or  Monitoring  Eisenhower-assisted  Activities 

(n=92) 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  24  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
directors  report  that  the  district  Eisenhower  coordinator  is  involved  in  planning,  implementing,  or  monitoring  the  project.  Each  bar  and 
the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each  category. 


Co-funding 

A final  dimension  of  coordination  is  co-funding,  or  the  extent  to  which  SAHE-grantee 
projects  coordinate  funding  with  other  programs  operating  in  the  districts  with  which  they  work.  For 
a number  of  National  Science  Foundation  and  ED  department  programs  (i.e.,  NSF’s  State  Systemic 
Initiative  (SSI),  Urban  Systemic  Initiative  (USI),  Rural  Systemic  Initiative  (RSI),  and  Local  Systemic 
Initiative  (LSI),  and  ED’s  Title  I,  Part  A program),  we  asked  each  SAHE  grantee  if  the  program 
operated  in  the  districts  with  which  the  IHE/NPO  worked  during  the  last  year,  and  if  the  program 
supports  professional  development.  We  also  asked  whether  the  program  co-funded  professional 
development  last  year  with  the  SAHE-grantee’ s project,  and  whether  the  program  staff  worked 
closely  with  the  grantee’s  staff  in  the  last  year.  Results  in  Exhibit  6. 19  show  that  SAHE  grantees  do 
little  co-funding  with  federal  programs,  although  they  report  high  levels  of  working  with  other 
programs.  (Note:  results  reported  for  working  closely  with  districts  are  contingent  upon  the  program 
existing  in  the  district  in  which  the  IHE/NPO  is  working;  and  co-funding  results  are  contingent  upon 
both  the  program’s  existence  in  the  district  and  the  program’s  inclusion  of  professional  development. 
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EXHIBIT  6.19 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Co-fund  and/or 
Work  Closely  with  Other  Federal  Programs  (When  the  Program  Operates  in  the 

District)  (n  varies) 


Federal  programs 


□Worked  closely  with 
■ Co-funded  with 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  for  those  teachers  participating  in  SAHE-grantee  projects  in  projects  that  work 
with  districts  in  which  the  SSI  program  is  operating,  46  percent  of  participations  are  in  projects  that  report  working  closely  with  the 
SSI  program.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each  category. 


Eighty-one  percent  of  participating  teachers  are  in  projects  whose  directors  say  they  work 
closely  with  the  RSI  program  staff.  To  a lesser  extent,  SAHE  grantees  work  with  staff  from  the  LSI 
(76  percent  of  participating  are  in  projects  whose  directors  work  with  LSI  staff),  the  USI  (62 
percent),  the  SSI  (46  percent),  and  Title  I,  Part  A (22  percent).  However,  while  SAHE  grantees  work 
with  staff  of  other  programs,  grantees  often  do  not  co-fund  with  them.  No  SAHE  grantees  co-fund 
with  the  Rural  Systemic  Initiative,  and  few  co-fund  with  the  USI  (15  percent  of  participating  teachers 
in  projects  located  in  states  with  USIs,  where  the  USIs  support  professional  development,  are  in 
projects  that  co-fund  with  the  USI).  Sixty-one  percent  of  participating  teachers  in  projects  in  districts 
with  LSIs  that  support  professional  development  are  in  projects  that  co-fund  with  the  LSI.  To  a 
lesser  extent,  participating  teachers  are  in  projects  that  co-fund  with  Title  A (37  percent)  and  the  SSI 
(27  percent). 

These  results  support  the  notion  that  many  IHE/NPOs  work  with  districts  only  after  the 
IHE/NPO  activity  has  been  designed.  In  order  for  co-funding  to  occur,  there  would  have  to  be  more 
coordination  in  the  design  phase  of  the  activity,  and  our  evidence  suggests  that  SAHE  grantees 
design  their  activities  independently  of  the  district.  Although  many  project  directors  report  working 
with  district  administrators,  this  may  occur  primarily  in  the  implementation  phase,  and  thus  not  have 
an  effect  on  the  design  or  structure  of  the  activities. 
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Overall  Coordination  with  Districts 


We  combined  all  five  of  these  scales  to  create  a coordination  composite  measuring  the  extent 
of  coordination  that  each  SAHE  grantee  has  with  districts.  There  are  12  possible  types  of 
coordination,  so  the  scale  ranges  from  0 to  12. 14  As  Exhibit  6.20  indicates,  projects  in  “other” 
departments  rarely  have  more  than  six  coordination  strategies;  projects  in  education  departments 
have  between  3 and  11;  and  projects  in  mathematics/science  departments  have  from  1 to  9 types  of 
coordination.  Exhibit  6.20  also  shows  that  the  level  of  coordination  does  not  vary  significantly  by 
institution  type.  However,  SAHE-supported  IHE  projects  in  education  departments  have 
significantly  more  coordination  than  IHE  projects  in  mathematics  or  science  departments  (6.5  types 
compared  to  4.8).  These  findings  offer  more  support  for  the  notion  that  education  departments  are 
more  closely  tied  to  districts  than  are  mathematics  or  science  departments. 

EXHIBIT  6.20 


Number  of  Types  of  Coordination  of  the  SAHE-grantee  Project  with  Districts,  Overall 
and  by  Institution  Type  and  Departmental  Affiliation  (n=92) 
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Significant  Pairwise  Contrasts 

Departmental  Affiliation Mathematics/Science  vs.  Education 


Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  on  average,  teachers  in  SAHE-grantee  projects  are  in  projects  that  report  having 
5.5  of  a possible  12  types  of  coordination  with  districts.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  1HE/NPO  projects  at 
one  data  point  (or  value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for 
that  particular  category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


14  The  extent  to  which  IHE/NPOs  work  closely  with  federal  programs  is  not  part  of  our  overall  coordination 
composite  because  the  intent  of  the  composite  is  to  measure  overall  coordination  efforts  with  districts,  not  federal- 
or  state-level  coordination. 
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Overall,  our  survey  data  indicate  that  districts  and  SAHE  grantees  work  together  in  some 
ways,  but  not  others,  and  that  district  goals  and  standards  are  not  always  integrated  into  SAHE- 
grantee  activities.  This  conclusion  is  supported  by  data  from  our  interviews  with  a number  of  SAHE- 
grantee  project  directors  that  we  conducted  in  conjunction  with  our  in-depth  case  studies.  For 
example,  in  Ohio,  a school  district  and  partnership  schools  in  the  city  are  involved  in  the  SAHE- 
sponsored  IHE  professional  development  activity,  which  focuses  on  providing  science  materials  and 
equipment  to  science  teachers.  The  university  provides  direction,  coordination,  and  evaluation  of  the 
project;  facilitates  the  assembly,  organization,  and  storage  of  teaching  kits,  hardware,  and  software; 
and  provides  technical  assistance.  The  university  also  is  involved  in  planning,  guiding  teachers’ 
academic  progress,  preparing  and  grading  exams,  and  lending  and  hauling  specialized  equipment  for 
demonstrations.  The  local  school  districts  provide  the  use  of  their  library  and  classrooms, 
audiovisual  equipment,  utilities,  and  janitorial  service  for  several  classroom  sessions.  All 
cooperating  schools  also  agreed  to  provide  $150  per  teacher-participant  for  instructional  materials  to 
be  used  with  their  students  in  local  classrooms.  In  addition,  one  of  the  teachers  from  the  district  is 
involved  in  every  planning  session  and  reports  personally  to  the  superintendent.  The  local  county 
Board  of  Education  provides  courier  service  and  some  clerical  help  and  time  for  communications, 
recruitment,  and  supervision  of  teacher  selection.  Despite  this  multi-level  collaboration,  the  IHE 
reports  that  their  project  does  not  relate  to  the  professional  development  goals  of  the  district,  and  in 
fact,  the  IHE  is  unfamiliar  with  the  district’s  professional  development  goals. 

Similarly,  in  Texas,  a SAHE-spon sored  IHE  project  director  collaborates  with  district  and 
federal  programs,  but  reports  that  district  standards  and  assessments  have  little  effect  on  the  project’s 
professional  development  activities.  The  project’s  primary  goal  is  to  create  a support  system  to 
increase  implementation  of  the  national  standards  in  science  and  math.  The  superintendent  or 
assistant  superintendent  of  the  district  must  sign  off  on  the  project  and  agree  that  their  teachers  will 
participate.  The  project  is  coordinated  with  the  USIs  in  the  districts  that  the  project  serves,  and  there 
is  collaborative  planning  with  the  districts’  science  coordinators,  three  USI  directors  from  two 
different  districts,  a grant  coordinator  from  one  of  the  districts,  and  professors  from  the  college.  The 
Eisenhower  and  USI  directors  volunteer  time  to  work  with  each  other.  Mentor  teachers  also  give 
input  into  the  project  development.  The  IHE  project  director  reports  that  she  works  very  closely  with 
the  district.  She  spends  approximately  200  hours  in  classrooms  each  year  doing  follow-up.  Despite 
this  level  of  coordination  and  collaboration,  the  director  reports  that  district  assessments,  standards, 
or  indicators  do  not  affect  the  design  or  implementation  of  the  program. 

Thus,  if  the  goal  of  requiring  SAHE  grantees  to  develop  cooperative  agreements  with 
districts  and  to  work  closely  with  districts  in  implementing  their  projects  is  to  foster  collaboration  in 
the  design  of  professional  development  activities,  alignment  with  district  standards  and  assessments, 
and  integration  with  district  programs  and  reforms,  IHEs  and  districts  do  not  seem  to  be  meeting  the 
goal.  While  IHE/NPOs  report  working  closely  with  districts  on  several  activities,  they  report  not 
working  closely  on  other  key  dimensions,  such  as  co-funding  and  working  with  the  district 
Eisenhower  coordinators.  While  the  independence  of  IHE/NPOs  may  serve  an  important  function  in 
allowing  them  to  provide  new  knowledge  and  innovative  approaches  to  teaching,  coordination  with 
districts  might  be  encouraged  in  specific  areas  where  it  may  benefit  both  IHE/NPOs  and  districts, 
such  as  in  co-funding  and  targeting. 
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Implementing  the  Vision:  IHE/NPO  Procedures 
for  the  Continuous  Improvement  of 
Professional  Development  Activities 


Section  Findings 

♦ For  SAHE  grantees  in  states  or  districts  with  indicators,  state  indicators  have  a much 
greater  effect  on  SAHE  grantees  than  district  indicators  do. 

♦ About  two-thirds  of  SAHE  grantees  conduct  needs  assessments  of  teachers;  the  most 
popular  methods  of  conducting  these  needs  assessments  are  teacher  surveys  and  informal 
conversations  with  teachers. 

♦ Almost  all  SAHE  grantees  evaluate  their  activities.  Of  those  that  evaluate,  almost  all  use 
teacher  surveys  as  a method  of  evaluation;  the  majority  also  use  counts  of  teacher 
participations  and  observations  of  teachers;  less  than  a third  use  student  achievement  as 
an  evaluation  measure. 

♦ Research/doctoral  universities  use  fewer  continuous  improvement  methods  than  other 
types  of  IHEs,  and  IHE  projects  housed  in  mathematics  or  science  departments  use 
significantly  fewer  continuous  improvement  methods  than  IHE  projects  housed  in  either 
education  or  other  (i.e.,  non-mathematics,  science,  or  education)  departments. 

In  addition  to  alignment  and  coordination,  another  method  of  shaping  professional 
development  activities  involves  the  use  of  indicators,  needs  assessments,  and  evaluations  to  help 
design  and  implement  activities.  As  we  noted  earlier,  the  legislation’s  overall  purposes  apply  to 
SAHE  grantees;  the  law  calls  for  SAHE  grantees  to  provide  professional  development  that  “creates 
an  orientation  toward  continuous  improvement  throughout  the  school”  (Section  2002(2)(F)).  The 
law  requires  that  SAHE  grantees  establish  performance  indicators  (Section  2208(a)(2)),  conduct  an 
assessment  of  teachers  needs  (Section  2208(b)(1)),  and  report  to  the  state  on  the  progress  toward 
meeting  the  indicators  (Section  2401(b)  and  Section  2208(d)(1)(G)).  These  provisions  reflect  the 
“continuous  improvement”  paradigm  that  the  federal  government  has  adopted,  embodied  by  the 
Government  Performance  and  Results  Act.  Research  has  suggested  that  such  continuous 
improvement  methods  are  important  for  high-quality  professional  development.  The  quality  of 
professional  development  increases  when  teachers  and  providers  are  held  accountable  for  outcomes 
of  professional  development  (Loucks-Horsley  et  al.,  1998)  and  when  professional  development  is 
evaluated  based  on  teacher  and  student  outcomes  (Guskey,  1997). 

We  examined  SAHE-grantee  continuous  improvement  efforts  by  asking  the  project  directors 
about  their  use  of  indicators,  needs  assessments,  and  evaluations  in  the  design  and  implementation  of 
their  professional  development  activities. 

Presence  and  Use  of  Performance  Indicators 

We  asked  SAHE-grantee  project  directors  whether  they  are  aware  of  any  performance 
indicators  for  professional  development  set  by  the  state  and/or  district,  and  if  so,  whether  these 
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indicators  affect  their  Eisenhower  projects.  Exhibit  6.21  illustrates  that  50  percent  of  teachers 
participating  in  SAHE-grantee  projects  are  in  projects  whose  directors  report  being  aware  of  state 
performance  indicators;  of  these,  77  percent  are  in  projects  whose  directors  also  report  that  the 
Eisenhower  project  is  affected  by  the  state  indicators.  Forty  percent  of  participating  teachers  are  in 
projects  that  work  with  districts  that  have  performance  indicators;  of  these,  only  22  percent  are  in 
projects  whose  directors  say  that  the  indicators  affect  the  project.15  Thus,  state  indicators  seem  to 
have  more  effect  on  SAHE-grantee  projects  than  do  district  indicators.  This  is  consistent  with 
findings  reported  in  Chapter  5,  which  show  that  districts  are  also  more  affected  by  state  than  district 
indicators,  and  with  findings  reported  earlier  in  this  chapter,  which  show  that  state  standards  and 
assessments  play  a larger  role  in  project  design  than  district  standards  or  assessments. 

EXHIBIT  6.21 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  by  Status  and  Effect  of 
District  and  State  Performance  Indicators  on  the  Project  (n=92) 


Status  of  state  and  district  performance  indicators 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit:  The  first  bar  shows  that  50  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that  report 
that  their  state  has  performance  indicators.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each 
category. 


In  our  case-study  interviews,  an  IHE  project  director  in  Kentucky  said  that  state  assessments 
have  affected  their  projects  more  than  district  assessments,  because  schools  in  the  state  are  held 
accountable  for  meeting  the  state  goals,  and  the  IHE  projects  aim  to  help  teachers  in  this  effort.  Also, 
the  IHE  director  said  that  district-level  indicators  do  not  affect  the  Eisenhower  project  directly, 
because  the  project  works  with  many  districts.  Nevertheless,  district  indicators  may  have  an  indirect 
influence,  because  they  are  usually  in  line  with  the  state  indicators.  An  IHE  project  director  in  Texas 


15  This  number  reflects  SAHE  grantees  that  report  that  either  some,  most,  or  all  of  the  districts  that  they  work  with 
have  performance  indicators. 
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supported  this  view,  reporting  that  district  assessments  have  not  affected  the  design  of  the 
Eisenhower  program  because  their  project  is  based  on  the  national  standards. 

Assessing  the  Needs  of  Teachers 

Another  dimension  of  continuous  improvement  is  assessing  teachers’  needs  in  order  to 
identify  potential  areas  for  improvement.  About  66  percent  of  participating  teachers  are  in  projects 
that  say  they  formally  assess  teachers’  needs  for  professional  development  (data  not  shown).  We 
asked  SAHE-grantee  project  directors  how  they  assess  teachers’  needs  for  professional  development. 
Options  were  1)  with  a survey  of  teachers,  2)  with  meetings  of  teacher  representatives,  3)  with  a 
survey  of  principals  or  department  chairs,  4)  with  measures  of  student  performance,  and  5)  with 
informal  conversations.  Exhibit  6.22  shows  that  the  most  common  methods  are  surveying  teachers 
(51  percent  of  participating  teachers  are  in  projects  that  use  this  method)  and  informal  conversations 
(50  percent).  Meetings  with  teacher  representatives  (41  percent)  are  less  frequent.  Less  than  a 
quarter  of  participating  teachers  are  in  projects  that  indicate  that  they  use  measures  of  student 
performance  (24  percent)  or  surveys  of  principals  or  department  chairs  (19  percent)  to  assess 
teachers’  professional  development  needs.  Thus  the  most  common  source  of  needs  assessment 
information  is  teachers.  While  it  is  important  to  consider  teachers’  perceptions  of  their  own  needs  as 
well  as  their  students’  needs,  it  may  also  be  useful  to  combine  this  with  information  from  other 
sources,  such  as  classroom  observations. 


EXHIBIT  6.22 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Use  Different 
Strategies  to  Assess  Teachers’  Professional  Development  Needs  (n=92) 


Meeting 

Needs  assessment  methods 


Source.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1 998, 

How  to  read  this  exhibit : The  first  bar  shows  that  5 1 percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  that 
report  using  teacher  surveys  as  a strategy  for  assessing  teachers’  professional  development  needs.  Each  bar  and  the  number  on  top  of 
it  represent  the  percent  of  participating  teachers  for  each  category. 


6-51 


Compared  to  the  SAHE-grantee  project  directors,  district  coordinators  report  much  higher 
levels  of  needs  assessments  of  every  kind  (see  Exhibit  5.12  in  Chapter  5).  There  are  several  possible 
explanations  for  the  higher  levels  of  needs  assessment  in  districts.  One  explanation  is  that,  as  shown 
in  Exhibit  6.17,  65  percent  of  participating  teachers  are  in  projects  that  rely  on  district  needs 
assessments  to  design  their  professional  development  program,  rather  than  designing  their  own. 

Other  reasons  may  be  that  districts  have  better  access  to  teachers  than  do  SAHE  grantees,  making 
assessment  methods  such  as  “informal  conversations”  much  easier  to  implement.  Districts  can  more 
easily  identify  the  pool  of  teachers  for  which  the  professional  development  will  be  designed,  whereas 
SAHE  grantees  may  be  working  with  several  districts  and  it  may  not  be  clear  from  where  participants 
will  be  drawn.  Further,  many  IHE-sponsored  activities  are  college  courses,  which  may  be  designed 
to  fulfill  needs  of  teachers  who  are  students  at  the  college,  rather  than  to  fulfill  the  needs  of  teachers 
in  districts  with  which  the  IHE  may  be  working.  Also,  college  professors  may  rely  on  theory  and 
research  to  identify  teachers’  needs,  rather  than  surveying  teachers  themselves. 

Evaluating  Professional  Development  Activities 

The  third  component  of  continuous  improvement  that  we  measured  is  evaluation  of  the 
professional  development  activity.  We  asked  SAHE  grantees  if  they  evaluate  their  Eisenhower- 
assisted  professional  development;  ninety-three  percent  of  participating  teachers  are  in  projects 
whose  project  directors  say  that  they  evaluate  these  activities.  We  then  asked  SAHE  grantees  how 
they  evaluate  their  professional  development.  Alternatives  include  1 ) by  the  number  of  teachers 
participating  in  professional  development,  2)  with  a teacher  satisfaction  survey,  3)  with  observations 
of  teachers,  and/or  4)  with  student  achievement.  As  Exhibit  6.23  illustrates,  the  most  common 
method  of  evaluation  is  a teacher  satisfaction  survey,  which  almost  all  SAHE  grantees  use;  ninety- 
two  percent  of  participating  teachers  are  projects  that  use  surveys.  To  a lesser  extent,  SAHE  grantee 
project  directors  report  using  a count  of  the  number  of  teachers  who  participate  (61  percent  of 
participating  teachers  are  in  projects  that  use  this  method)  and  observations  of  teachers  (59  percent) 
as  methods  for  evaluating  their  Eisenhower-assisted  professional  development  activities.  The  least 
common  method  of  evaluation  is  using  student  achievement  as  a measure  of  the  professional 
development  activity;  3 1 percent  of  participating  teachers  are  in  projects  that  use  this  method. 

One  IHE  project  director  in  Texas  offered  a possible  explanation  for  the  relatively  infrequent 
use  of  student  achievement  as  a method  of  evaluating  professional  development.  She  explains  that 
science  is  addressed  infrequently  on  state  tests  and  therefore  the  only  regular  indicator  of  student 
achievement  in  science  that  is  available  is  student  grades.  This  may  explain  why  student 
achievement  measures  are  not  used  for  science,  but  it  would  not  explain  why  they  are  not  used  for 
mathematics.  It  may  be  that  IHE/NPOs  do  not  use  student  achievement  measures  because  of  the 
complexity  involved  in  identifying  comparable  measures  over  time  and  separating  the  effects  of 
professional  development  from  other  effects.  Using  student  achievement  to  determine  effects  of 
professional  development  is  a complex,  long-term  undertaking  involving  longitudinal  studies  of  , 
teachers  and  students;  SAHE  grantees  may  not  have  the  resources  or  capacity  to  launch  such 
evaluations.  Given  time,  expertise,  and  resource  constraints,  a more  appropriate  method  of 
evaluation  is  for  SAHE  grantees  to  evaluate  their  professional  development  activities  based  on  the 
characteristics  of  high  quality  that  we  have  shown  in  this  report  to  be  indicative  of  effective 
professional  development.  For  example,  SAHE  grantees  could  document  the  duration  of  their 
activities,  and  the  opportunities  for  active  learning  and  collective  participation.  In  addition,  SAHE 
grantees  might  evaluate  the  extent  to  which  participants  gain  the  intended  knowledge  and  skills  or 
show  improvements  in  classroom  teaching. 
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IXHIBBT  6.23 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Use  Different 
Methods  for  Evaluating  Activities  (n=92) 


Participation  Achievement 

Evaluation  methods 


Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  bar  shows  that  61  percent  of  teachers  participating  in  SAHE-grantee  projects  are  in  projects  whose 
director  reports  that  counts  of  teacher  participation  are  used  as  a method  for  evaluating  Eisenhower-assisted  professional  development 
activities.  Each  bar  and  the  number  on  top  of  it  represent  the  percent  of  participating  teachers  for  each  category. 

Overall  Continuous  Improvement 

Taken  together,  indicators,  needs  assessments,  and  evaluation  strategies  can  be  components 
of  a process  of  continuous  improvement  in  the  design  of  professional  development.  To  examine 
whether  SAHE  grantees’  use  of  continuous  improvement  methods  differs  significantly  by  institution 
type  or  affiliation,  we  created  scales  for  indicators,  needs  assessments,  and  evaluation,  rescaled  each 
to  have  a range  from  0 to  1,  then  created  an  additive  composite  of  all  three  scales,  with  a range  of  0 
to  3.  Exhibit  6.24  shows  the  results  for  this  continuous  improvement  composite.  It  indicates  great 
variation  in  the  extent  to  which  SAHE  grantees  use  continuous  improvement  strategies.  The  data 
also  indicate  that  nonresearch/doctoral -granting  institutions  use  significantly  more  continuous 
improvement  strategies  than  do  research/doctoral-granting  institutions.  As  described  earlier, 
research/doctoral-granting  institutions  may  be  more  likely  to  design  their  Eisenhower  projects  to 
reflect  current  research  agendas  of  the  faculty,  and  therefore  may  be  less  likely  than  other  types  of 
colleges  and  universities  to  work  with  districts  to  provide  activities  tailored  to  the  needs  of  their 
teachers,  which  would  require  more  needs  assessments  and  evaluation.  Our  data  also  show  that  IHE 
projects  housed  in  education  departments  engage  in  significantly  more  continuous  improvement 
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activities  than  IHE  projects  housed  in  mathematics  or  science  departments.  As  we  discussed  earlier 
in  the  chapter,  professors  in  education  departments  may  have  closer  relationships  with  districts  than 
mathematics  or  science  professors.  This  in  turn  may  increase  their  access  to  teachers  in  the  district, 
for  purposes  of  getting  feedback  on  how  professional  development  activities  might  apply  to  district 
indicators,  and  for  gaining  access  to  teachers  to  assess  needs  and  follow-up  with  evaluative  measures 
Also,  education  professors  are  more  likely  than  mathematics  or  science  professors  to  be  social 
scientists  and  to  use  social  science  methods  upon  which  the  process  of  continuous  improvement  is 
based. 


EXHIBIT  6.24 


Extent  of  SAHE-grantee  Continuous  Improvement  Efforts,  Overall  and  by 
Institution  Type  and  Departmental  Affiliation  (n=92 ) 
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Significant  Pairwise  Contrasts 

Departmental  Affiliation Mathematics/Science  vs.  Education;  Mathematics/Science  vs.  Other 


Source.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

How  to  read  this  exhibit : The  first  distribution  shows  that  teachers  in  SAHE-grantee  projects  are  in  projects  that  report  an  average 
continuous  improvement  score  of  1 .4,  where  zero  indicates  no  continuous  improvement  efforts  and  three  indicates  the  largest  extent  of 
continuous  improvement  efforts.  Each  dot  represents  one  IHE/NPO  project.  As  the  number  of  1HE/NPO  projects  at  one  data  point  (or 
value)  increases,  the  dots  form  a horizontal  line  that  increases  in  length.  Each  distribution  represents  the  distribution  for  that  particular 
category.  The  number  to  the  right  of  the  distribution  is  the  mean. 


Summary:  Continuous  Improvement 

Our  data  from  SAHE-grantee  project  directors  show  that  grantees  are  much  more  likely  to 
use  state  indicators  than  district  indicators.  This  parallels  our  finding  reported  earlier  in  this  chapter 
that  SAHE-grantee  projects  are  more  likely  to  be  aligned  with  state  than  district  standards  and 
assessments.  These  findings  may  indicate  that  the  SAHE  competitive-proposal  process  helps  to 
ensure  that  SAHE-grantee -provided  professional  development  is  linked  to  statewide  reform.  But  the 
results  also  imply  that  the  law’s  requirements  that  SAHE  grantees  develop  cooperative  agreements 
and  work  closely  with  the  district  do  not  translate  into  the  use  of  district  standards,  assessments,  and 
indicators.  Further,  we  find  that  while  most  SAHE-grantee  project  directors  conduct  needs 
assessments  and  evaluations,  these  processes  typically  are  based  on  some  form  of  feedback  from 
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teachers,  such  as  surveys  or  informal  conversations.  Observations  of  teachers  or  measures  of  student 
achievement,  which  may  be  more  objective  ways  of  assessing  needs  and  evaluating  outcomes,  are 
less  commonly  used  for  these  purposes.  This  may,  in  part,  be  due  to  the  difficulty  of  obtaining  these 
measures,  especially  student  achievement.  Finally,  analysis  of  our  continuous  improvement 
composite  indicates  that  projects  in  research/doctoral  institutions  and  mathematics/science 
departments  use  continuous  improvement  processes  significantly  less  than  other  institutions  and 
education  departments,  respectively. 


Differences  in  SAHE-Grantee  Projects  by  Institution  Type 

and  Departmental  Affiliation 


Throughout  this  chapter,  we  have  examined  whether  the  features  and  management  of 
Eisenhower-assisted  professional  development  differ  by  type  of  institution  and  departmental 
affiliation.  With  respect  to  type  of  institution,  we  find  that,  compared  to  nonresearch/doctoral- 
granting  institutions,  research/doctoral-granting  institutions: 

♦ have  significantly  more  collective  participation  in  their  primary  activity; 

♦ are  less  aligned  with  state  and  district  standards  and  assessments;  and 

♦ use  fewer  continuous-improvement  methods. 

With  respect  to  departmental  affiliation,  our  data  indicate  that,  compared  to  projects  in 
mathematics/science  departments,  projects  in  education  departments:16 

♦ are  more  likely  to  be  reform  rather  than  traditional; 

♦ sponsor  professional  development  activities  that  span  a longer  time  period; 

♦ engage  in  more  types  of  coordination  with  the  district;  and 

♦ use  more  continuous  improvement  efforts  (“other”  departments  also  use  significantly 
more  continuous  improvement  efforts  than  mathematics  or  science  departments). 

Thus,  our  data  suggest  that  project  directors  who  are  affiliated  with  education  departments 
design  and  implement  Eisenhower  professional  development  projects  that  are  superior  to  those 
designed  by  directors  associated  with  mathematics  or  science  departments,  on  a number  of 
dimensions. 

Further,  we  find  some  evidence  of  interactions  between  institution  type  and  departmental 
affiliation. 


16  Although  a project  that  is  housed  in  a particular  department  may  be  administered  collaboratively  by  the  school  of 
education  and  the  college  or  school  in  which  the  mathematics/science  department  resides,  our  study  focuses  on  the 
departmental  affiliation  of  the  project  director. 
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♦ Projects  in  mathematics/science  departments  have  high  content  focus  regardless  of  the 
type  of  IHE  they  are  in,  while  projects  in  education  and  other  departments  have  a high 
content  focus  only  if  they  are  in  research/doctoral  universities. 

♦ Projects  in  education  departments  have  many  opportunities  for  active  learning  when 
housed  in  research  universities,  but  only  a moderate  number  when  housed  in  nonresearch 
universities.  The  opposite  is  true  for  mathematics/science  department  projects — they 
have  more  active  learning  opportunities  in  nonresearch  than  research  universities. 

♦ Projects  in  “other”  departments  have  a high  number  of  active  learning  opportunities 
regardless  of  the  type  of  institution  they  are  in. 

Opportunities  for  active  learning  and  number  of  contact  hours  are  greater  in  education 
departments,  but  only  in  education  departments  at  research/doctoral -granting  institutions.  This  may 
indicate  that  education  professors  in  research  institutions,  compared  to  other  institutions,  are  more 
likely  to  be  familiar  with  recent  professional  development  literature  that  emphasizes  the  importance 
of  active  learning  opportunities  and  extended  contact  hours.  Further,  education  professors  in 
research  universities  are  the  primary  source  of  research  in  professional  development,  and  are  the 
primary  promoters  of  reforms  in  the  quality  of  activities;  therefore  it  is  not  surprising  that  these 
professors  are  the  ones  most  likely  to  direct  projects  with  high-quality  structural  and  core  features. 
This  also  suggests  a potentially  important  distinction.  In  general,  perhaps  due  to  the  goals  that  a 
department  that  trains  teachers  has  in  common  with  the  school  district  that  it  serves,  professors 
associated  with  education  departments  may  have  a closer  relationship  with  districts  than  mathematics 
or  science  professors.  This  closer  relationship  creates  a natural  communication  that  permits  closer 
collaboration,  which  in  turn  may  foster  more  targeting,  coordination,  and  continuous-improvement 
efforts.  This  reasoning  may  also  help  to  explain  why  research  universities  score  lower  in  continuous 
improvement  and  alignment  than  other  types  of  institutions.  IHEs  that  have  larger  teacher  education 
programs,  which  are  more  likely  to  be  institutions  that  grant  only  bachelor’s  and  master’s  degrees, 
are  likely  to  have  closer  ties  to  local  school  districts.  These  ties  facilitate  the  sharing  of  information 
and  collaboration  that  is  necessary  for  continuous-improvement  and  alignment  efforts.  If 
research/doctoral-granting  universities  are  less  likely  to  have  these  links  to  the  district,  projects  in 
these  universities  would  be  less  likely  to  implement  continuous-improvement  and  alignment 
strategies.  Also,  project  directors  in  research  universities  are  more  likely  to  have  their  own  research 
agendas  that  help  shape  the  activities  that  they  design,  whereas  their  colleagues  at  nonresearch 
universities  may  be  more  free  to  be  responsive  to  the  district’s  needs. 

The  high  degree  of  content  focus  of  projects  in  mathematics/science  departments  in  either 
type  of  institution  is  not  a surprise,  given  that  mathematics/science  departments  usually  do  not 
include  pedagogy  and  processes  in  their  curriculum,  as  education  departments  do.  The  higher 
content  focus  of  projects  in  education  and  other  departments  in  research  universities,  compared  to 
projects  in  nonresearch  universities,  may  indicate  that  project  directors  in  nonresearch  institutions 
need  more  knowledge  about  the  importance  of  content  focus  in  professional  development  activities. 
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The  Relationship  of  SAHE-Grantee  Management  to 
Features  of  Professional  Development 


So  far  in  the  chapter  we  have  examined  the  features  of  SAHE-grantee  activities,  and  how 
SAHE  grantees  manage  and  operate  their  professional  development  activities.  We  now  look  at  the 
associations  among  these  variables,  to  examine  how  they  work  together  and  affect  each  other. 

Exhibit  6.25  shows  the  relationships  among  the  implementation  and  quality  variables  that  we  discuss 
in  this  chapter.  (For  a more  detailed  presentation  of  the  results,  see  Appendix  H.)  All  reported 
coefficients  are  standardized  path  coefficients,  which  represent  the  influence  of  one  variable  on 
another.  The  range  in  values  for  path  coefficients  is  generally  between  -1  and  1,  and  greater  absolute 
values  represent  stronger  relationships.  All  paths  shown  are  statistically  significant  at  the  .05  level. 

The  model  is  designed  to  reflect  an  implied  causal  structure:  variables  on  the  far  left  are 
considered  to  be  exogenous  variables,  or  external  to  the  system,  since  they  are  preconditions  and  are 
not  manipulate  (except  to  the  extent  that  grants  can  be  targeted  to  particular  types  of  institutions  or 
departments  within  institutions).  Alignment  and  coordination  are  implementation  variables  that  are 
posited  to  affect  structural  and  core  features  of  professional  development  activities  both  directly  and 
indirectly  through  increased  continuous-improvement  efforts. 


Exhibit  6.25  suggests  that  it  is  in  part  through  coordination  with  districts  that  education 
departments  outperform  mathematics/science  departments  on  many  dimensions  of  professional 
development.  Coordination  with  the  district  is  associated  with  several  structural  and  core  features. 
Projects  that  coordinate  with  districts  are  less  likely  to  have  reform  activities  or  activities  with  long 
time-spans,  but  more  likely  to  have  activities  with  collective  participation  and  a greater  number  of 
contact  hours.  This  might  suggest  that  if  an  activity  is  coordinated  with  the  district,  it  might  be 
planned  around  the  school  year,  and  occur  during  breaks  or  the  summer,  which  would  require  a 
shorter  span  of  time,  but  allow  more  contact  hours.  One  would  also  expect  more  collective 
participation,  since  the  project  director  in  the  education  department  is  working  closely  with  the 
district,  perhaps  planning  professional  development  to  meet  the  needs  of  particular  groups  of  teachers 
or  schools. 

The  model  also  indicates  that  SAHE  grantees  that  engage  in  coordination  make  greater  use  of 
continuous  improvement  strategies  (i.e.,  indicators,  needs  assessments,  and  evaluation),  and  in  turn 
are  more  likely  to  offer  more  active  learning  opportunities  and  target  specific  groups  of  teachers, 
perhaps  in  response  to  the  needs  assessments  and  evaluations  provided  by  teachers.  Alternatively, 
SAHE  grantees  inclined  to  practice  coordination  and  continuous-improvement  efforts  may  already  be 
reform-minded,  and  thus  inclined  to  design  their  activities  with  many  active  learning  strategies,  and 
to  reach  teachers  of  at-risk  students. 

In  contrast  to  education  departments,  nonresearch/doctoral-granting  institutions,  NPOs,  and 
mathematics/science  departments  have  no  positive  paths,  and  some  negative  paths,  to  desirable 
characteristics  of  professional  development.  In  particular,  mathematics/science  departments  are 
much  less  likely  to  engage  in  continuous  improvement  strategies,  and  thus  have  projects  with  fewer 
opportunities  for  active  learning  and  less  targeting.  Further,  although  our  earlier  analysis  showed 
that  nonresearch/doctoral-granting  institutions  are  more  likely  than  research  institutions  to  align  their 
activities  with  state  and  district  standards  and  assessments,  alignment  does  not  prove  to  have  any 
associations  with  continuous  improvement  or  quality  features. 
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This  selected  analysis  of  our  SAHE-grantee  model  illustrates  the  importance  of  coordination 
and  continuous  improvement  in  determining  the  structural  and  core  features  of  professional 
development  activities.  These  results  suggest  that  the  emphasis  of  the  Title  II  legislation  on  SAHE- 
grantee  coordination  with  districts  is  well-placed,  considering  the  strong  relationship  that 
coordination  has  with  the  quality  of  professional  development,  and  with  continuous-improvement 
efforts.  Similarly,  the  law’s  attention  to  continuous-improvement  efforts  in  the  use  of  indicators, 
needs  assessment,  and  evaluation  is  also  well-supported  by  our  findings.  But,  as  reported  earlier  in 
this  chapter,  we  do  not  find  much  evidence  of  coordination  as  is  called  for  in  the  law.  Greater 
compliance  with  the  law  might  improve  the  overall  quality  of  the  professional  development  activities 
that  SAHE  grantees  provide. 

It  would  be  helpful  to  identify  the  factors  that  facilitate  coordination  and  continuous 
improvement  efforts  in  education  departments,  and  how  these  efforts  translate  into  higher  quality 
professional  development.  This  information  would  better  enable  us  to  apply  these  principles  and 
lessons  to  professional  development  provided  by  other  departments,  and  by  school  districts  and 
schools. 


In  this  chapter,  we  identified  the  types  of  SAHE  grantees  that  provide  professional 
development  through  the  Eisenhower  program;  described  the  structural  and  core  features  of  the 
professional  development  activities;  the  alignment,  coordination,  and  continuous-improvement 
efforts  of  SAHE  grantees;  and  examined  how  these  characteristics  might  be  explained  by  the  type  of 
institution  and  the  departmental  affiliation  of  the  project  director.  Several  of  these  findings  have 
implications  for  the  Eisenhower  program. 

First,  we  find  that  while  SAHE-sponsored  IHE/NPOs  tend  to  offer  traditional  types  of 
professional  development,  the  activities  have  many  contact  hours  and  span  many  months.  Second, 
SAHE-sponsored  IHE/NPOs  generally  provide  activities  with  strong  content  focus  and  many 
opportunities  for  active  learning.  These  characteristics  all  represent  characteristics  of  high-quality 
professional  development,  as  posited  in  the  literature  and  supported  by  our  data  in  Chapter  3. 
Although  there  is  variation  in  the  quality  of  SAHE-grantee  projects,  and  they  have  few  opportunities 
for  teachers’  collective  participation,  in  general  the  activities  they  provide  support  Title  II’s  goal  of 
providing  “sustained  and  intensive”  high-quality  professional  development  activities.  The  SAHE 
competitive  award  process  may  foster  high-quality  projects  through  establishing  criteria,  requiring 
projects  to  have  particular  characteristics,  and  monitoring  to  ensure  implementation;  but  we  do  not 
have  sufficient  data  on  these  SAHE-sponsored  competitions  to  examine  the  extent  to  which  these 
factors  affect  project  characteristics  and  operations. 

Third,  SAHE-grantee  project  directors  report  that  they  target  teachers  of  special  populations 
of  students  to  participate  in  professional  development  activities,  but  our  data  from  teachers,  reported 
in  Chapter  3,  show  that  the  actual  participation  of  these  groups  of  teachers  in  IHE  activities  is  low 
when  compared  to  participation  levels  in  district-sponsored  activities.  Given  Title  II’s  emphasis  on 
targeting  and  recruiting  teachers  of  diverse  students,  it  may  be  desirable  to  work  with  IHE/NPOs  to 
improve  their  methods  for  targeting  and  recruiting  these  teachers. 


Summary  and  Conclusions 
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Fourth,  SAHE  grantees  are  less  likely  to  co-fund  and  work  with  the  Eisenhower  coordinator 
than  they  are  to  communicate  or  work  closely  with  other  district  staff.  Coordination  with  the  district, 
which  includes  both  co-funding  and  working  with  the  district  in  many  other  ways,  is  one  of  the  most 
important  variables  in  our  model  of  quality  professional  development.  Also,  despite  high  levels  of 
coordination  in  some  areas,  SAHE-grantee  projects  tend  not  to  be  aligned  with  district  standards  and 
assessments  or  use  district  indicators;  they  respond  more  to  state  standards  and  indicators.  As  we 
suggested  earlier,  this  may  be  because  SAHE  grantees  sometimes  coordinate  with  districts  in  the 
implementation  of  activities,  but  not  in  their  development  and  design.  It  would  be  helpful  to 
understand  the  mechanisms  through  which  coordination  affects  the  quality  of  professional 
development,  so  that  we  could  draw  lessons  about  what  types  of  interactions  are  useful  for  shaping 
high-quality  professional  development,  and  how  they  work.  Perhaps  it  would  be  helpful  to  provide 
more  specific  guidelines,  emphasis,  and/or  training  for  particular  types  of  coordination  that  have  been 
shown  to  be  associated  with  the  provision  of  high-quality  professional  development. 

Fifth,  as  with  districts,  SAHE  grantees  tend  to  use  teacher  reports  and  surveys  rather  than 
classroom  observation  or  student  achievement  measures  to  assess  needs  and  evaluate  outcomes. 
Given  the  complexity  of  using  student  achievement  to  measure  the  effects  of  professional 
development,  it  may  be  desirable  to  emphasize  in  the  legislation  the  importance  of  evaluating  the 
quality  of  professional  development  activities  based  on  the  activities’  structural  and  core  features,  as 
described  in  the  literature  and  in  this  evaluation. 

Finally,  our  analyses  of  differences  by  institution  type  and  departmental  affiliation 
indicate  that,  while  projects  in  mathematics/science  departments  have  a high  content  focus, 
education  departments  do  better  than  mathematics/science  departments  on  several  dimensions  of 
quality  and  implementation.  As  our  model  illustrates,  this  is  probably  due  in  large  part  to  the 
better  coordination  that  education  departments  have  with  districts,  and  their  superior  continuous- 
improvement  efforts.  It  would  be  informative  to  get  a more  in-depth  sense  of  the  process  of 
coordination  that  take  place  between  education  departments  and  districts,  and  how  continuous- 
improvement  efforts  are  integrated  into  the  design  and  implementation  of  their  professional 
development  activities.  This  would  help  us  to  identify  the  specific  factors  and  processes  that  are 
most  influential  in  shaping  high-quality  professional  development  activities.  In  addition,  it  might 
be  useful  for  the  law  to  give  more  emphasis  to  the  desirability  of  having  mathematics  and  science 
departments  collaborate  with  education  departments  in  the  design  and  coordination  of  their 
Eisenhower  project. 
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CHAPTER  7 


CONCLUSIONS  AND  LESSONS  FOR  THE 
EISENHOWER  PROGRAM 


This  is  the  second  report  of  the  National  Evaluation  of  the  Eisenhower  Professional 
Development  Program,  Part  B (Title  II  of  the  Elementary  and  Secondary  Education  Act).  The  first 
report  was  based  on  six  exploratory  case  studies  of  school  districts  conducted  during  the  first  months 
of  the  evaluation  (Birman,  Reeve,  & Sattler,  1998).  The  purpose  of  that  report  was  to  obtain  an 
initial  description  of  the  program  and  the  issues  that  it  faced  in  different  local  contexts.  This  second 
report  describes  the  current  status  of  the  program,  based  on  data  from  national  probability  samples  of 
districts,  State  Agency  for  Higher  Education  (SAHE)  grantees,  and  teachers,  as  well  as  on  data  from 
10  in-depth  case  studies  in  five  states.  The  third  report  of  the  evaluation,  to  be  issued  in  early  2000, 
will  augment  the  cross-sectional  results  reported  here  with  a longitudinal  account  of  teachers' 
experiences  in  Eisenhower  and  other  professional  development  activities  and  changes  in  their 
teaching  practices. 

The  Eisenhower  Professional  Development  Program  is  the  U.S.  Department  of  Education's 
largest  investment  that  is  solely  focused  on  improving  the  quality  of  teaching  practice.  Part  B of  the 
Eisenhower  Professional  Development  program,  with  a 1999  appropriation  of  approximately  $335 
million,  provides  funds  through  states  to  school  districts  and  institutions  of  higher  education  and 
nonprofit  organizations.  These  funds  primarily  support  professional  development  in  mathematics 
and  science. 

In  its  1994  reauthorization  of  the  program,  Congress  made  explicit  that  Eisenhower-assisted 
professional  development  activities  should  be  designed  to  improve  teacher  practice  and,  ultimately, 
student  performance.  The  law  also  places  particular  emphasis  on  serving  teachers  in  schools  with 
high  poverty  rates.  Furthermore,  the  reauthorized  Eisenhower  program  embodies  policymakers' 
intention  that  the  program  support  systemic  education  reform  and  deeper  learning  among  teachers. 

In  the  analyses  conducted  for  this  report,  we  examine  the  extent  to  which  these  intentions 
have  been  achieved.  Our  conclusions  about  the  effectiveness  of  the  Eisenhower  program  are  divided 
into  four  main  parts.  First,  we  ask,  what  is  the  nature  of  current  classroom  teaching  practices  in 
mathematics  and  science,  and  what  directions  does  current  classroom  practice  suggest  for 
professional  development  (Chapter  2)?  Second,  what  evidence  do  we  find  that  teachers  experience 
high-quality  professional  development  in  Eisenhower-assisted  activities  (Chapter  3)?  Third,  what 
evidence  do  we  find  that  districts  develop  their  portfolios  of  professional  development  activities  and 
manage  and  operate  their  projects  in  ways  consistent  with  the  intent  of  the  legislation  (Chapters  4 and 
5)?  Fourth,  how  well  do  SAHE  grantee  projects  perform  with  respect  to  the  requirements  and  intent 
of  the  legislation  (Chapter  6)? 

These  major  areas  correspond  to  the  main  elements  in  our  conceptual  framework  for  the 
evaluation  (see  Exhibit  7.0). 
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EXHIBIT  7.0 


Conceptual  Framework  for  This  Evaluation 


Context  for  Eisenhower-assisted  Activities 
(District  Size  and  Poverty;  SAHE-grantee  Features) 


District  and  SAHE- 
grantee  Management  of 
Eisenhower-assisted 
Activities 

Buildiug  a Vision: 
Alignment  and 
Coordination 

Implementation: 

Continuous  Improvement 
and  Planning 


Source:  National  Profile 
(District  Coordinator  and 
SAHE-grantee 
Interviews)  and  Case 
Studies 


District  and  SAHE- 
grantee  “Portfolios”  of 
Eisenhower-assisted 
Professional 
Development  Activities 

Portfolio  Features 

Teacher  Recruitmeut 
and  Selection 


Source:  National 
Profile  (District 
Coordinator  and  SAHE- 
grantee  Interviews)  and 
Case  Studies 


Teacher  Experiences  in 
Eisenhower-assisted 
Professional  Development 


Source:  National  Profile 
(Teacher  Activity  Survey) 
and  Case  Studies 


Teaching  Practice 

* * * * 

Source:  Longitudinal 
Study  of  Teacher 
Change  and  Case  Studies 


Chapter  5 (District) 
Chapter  6 (SAHE  Grantee) 


Chapter  4 (District) 
Chapter  6 (SAHE  Grantee) 


Chapter  3 


Chapter  2 


In  the  following  section,  we  briefly  review  the  data  on  which  our  evaluation  is  based.  Then, 
we  turn  to  the  main  conclusions  and  implications  of  the  study. 


Strengths  of  the  Data 


Before  considering  the  findings  of  the  National  Evaluation  of  the  Eisenhower  Program,  it  is 
important  to  take  stock  of  the  quality  of  the  data  on  which  the  findings  rest.  The  data  set  has  a 
number  of  important  strengths.  First,  our  descriptions  of  the  nature  and  quality  of  professional 
development  provided  through  the  Eisenhower  program  are  based  on  national  probability  samples, 
each  with  an  excellent  response  rate.  The  national  probability  sample  of  district  programs  and  SAHE 
grantees  has  an  88  percent  response  rate  for  district  program  coordinators  and  87  percent  for  SAHE 
grantees.  The  national  probability  sample  of  teachers  who  participated  in  Eisenhower  professional 


development  activities  has  a response  rate  of  72  percent.  The  72  percent  response  rate  is  especially 
high  when  considering  the  multistage  process  necessary  to  complete  the  sample.1 

Second,  the  two  probability  samples  are  complementary.  Data  from  our  telephone  interviews 
with  district  program  coordinators  and  SAHE-grantee  project  directors  are  backed  up  by  teacher 
participant  accounts  of  what  they  experienced  and  its  quality. 

Finally,  we  have  taken  a number  of  steps  to  maximize  the  validity  and  reliability  of  the 
national  survey  data.  For  example,  although  the  telephone  interview  and  teacher  survey  data  are 
based  on  self-reports,  most  of  the  data  represent  an  accounting  of  behaviors,  not  direct  judgments  of 
quality  that  might  be  more  likely  biased  in  a positive  direction.  In  addition,  the  survey  results  are 
cross-validated  through  case  study  data  that  are  rich  in  potential  to  explain  the  descriptive  statistics 
and  path  analyses  from  the  surveys. 

The  three  strands  of  the  evaluation  are  designed  to  produce  an  integrated  portrait  of  the 
Eisenhower  program  from  many  perspectives.  Because  the  evaluation  involves  a variety  of  research 
methods  and  has  collected  data  from  groups  of  individuals  who  view  Eisenhower-assisted  activities 
from  different  vantage  points,  it  is  able  to  provide  an  accurate  description  of  program-funded 
activities  and  analyses  of  the  features  of  these  activities  and  their  effects  on  teacher  practice. 

The  data  we  analyze  in  this  report  do  not  provide  direct  estimates  of  changes  in  teaching 
practice  over  time.  Our  conclusions  about  the  effectiveness  of  Eisenhower-assisted  professional 
development  are  based  on  teachers’  reports  of  the  extent  to  which  participation  enhanced  their 
knowledge  and  skills  and  improved  their  teaching.  Data  from  the  second  and  third  waves  of  our 
longitudinal  study,  to  be  examined  in  our  third  report,  will  provide  additional  information  on  teacher 
change.2 

None  of  our  analyses  address  directly  how  teacher  participation  in  Eisenhower  professional 
development  leads  to  gains  in  student  achievement.  The  work  reported  here,  however,  is  not 
completely  divorced  from  student  achievement.  The  characteristics  of  professional  development  on 
which  project  directors  and  teacher  participants  reported  in  the  surveys  are  carefully  grounded  in  the 
available  literature  on  professional  development  and  student  achievement.  To  the  extent  that  the 
Eisenhower  program  provides  professional  development  with  characteristics  identified  as  effective  in 
the  literature,  we  conclude  that  they  are  of  high  quality,  and  we  tentatively  infer  that  they  should  lead 
to  benefits  for  students. 


1 District  coordinators  and  SAHE-grantee  project  directors  had  to  submit  the  complete  list  of  professional 
development  activities  provided  during  the  prior  year  and  the  number  of  participants.  Two  activities  were  selected 
from  each  district  with  probability  in  proportion  to  size,  and  from  those,  complete  rosters  of  teachers  were  collected 
from  which  two  teachers  were  randomly  selected  and  surveyed. 

2 We  also  took  a number  of  steps  to  maximize  the  validity  and  reliability  of  the  evaluation's  national  survey  data. 
For  example,  most  of  the  survey  questions  ask  teachers  and  administrators  to  provide  an  accounting  of  behaviors, 
not  direct  judgments  of  quality  that  might  be  more  likely  to  be  biased.  The  substantial  variation  in  the  responses 
teachers  and  district  administrators  provided  to  these  behavioral  items,  as  well  as  the  consistency  in  teacher  and 
district  administrator  responses,  tends  to  bolster  our  confidence  in  the  validity  of  the  data. 
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Current  Teaching  Practices  in  Mathematics  and  Science 


The  first  analyses  presented  in  this  report  focus  on  what  we  have  learned  from  the  baseline 
wave  of  our  longitudinal  survey  about  teachers’  classroom  practices  in  mathematics  and  science  at 
the  elementary,  middle,  and  high  school  levels.  The  data  for  these  baseline  analyses,  which  are 
presented  in  Chapter  2,  are  from  teachers  in  30  schools,  across  10  districts  in  five  states.  Teachers 
reported  on  the  content  they  taught  and  their  pedagogical  practices  for  the  1996-97  school  year.  In 
addition,  as  part  of  our  case  studies,  we  conducted  classroom  observations  of  two  teachers  in  each  of 
the  30  schools. 

These  data  allow  us  to  identify  weaknesses  in  classroom  practices  that  might  appropriately  be 
addressed  by  Eisenhower-assisted  professional  development  activities.  One  standard  we  used  to 
judge  the  quality  of  teacher  classroom  practices  is  the  degree  of  alignment  with  the  National 
Assessment  of  Education  Progress  (NAEP).  The  content  tested  on  the  NAEP  is  one  appropriate 
standard  against  which  to  judge  the  quality  of  content  offered  in  our  nation's  classrooms  because  the 
NAEP  is  considered  to  be  the  nation's  report  card.  The  National  Education  Goals  Panel  uses  the 
NAEP  to  monitor  progress  against  our  national  education  goals.  The  NAEP  is,  of  course,  not  without 
its  critics.  Some  would  like  the  NAEP  to  be  more  aligned  with  the  National  Council  of  Teachers  of 
Mathematics  (NCTM)  mathematics  standards  and  National  Research  Council  (NRC)  science  content 
standards.  Others  would  like  the  NAEP  to  place  a heavier  emphasis  upon  what  they  see  as  more 
basic  skills. 

To  allow  comparisons  to  the  NAEP,  we  obtained  all  mathematics  and  science  items  for 
NAEP  assessments  in  grades  4,  8,  and  12.  We  then  conducted  content  analyses  of  these  items  using 
the  same  descriptors  of  content  that  were  used  in  the  questionnaires  completed  by  teachers.  A useful 
byproduct  of  this  work  is  the  description  of  the  content  tested  on  the  NAEP  mathematics  and  science 
assessments.  The  two-dimensional  content  maps  we  prepared,  focusing  on  the  topics  and  student 
performance  goals  tapped  by  the  NAEP  items,  provide  an  easily-interpreted  picture  of  the  content 
emphasized  on  the  NAEP,  as  well  as  the  content  not  tested. 

Our  findings  on  current  teaching  practice  are  consistent  with  other  research,  especially  the 
Third  International  Mathematics  and  Sciences  Study  (TIMSS).  We  find  that  teachers  in  our  30  case 
study  schools  cover  a broader  range  of  content,  in  less  depth  than  is  reflected  in  the  NAEP.  Teachers 
might  be  characterized  as  teaching  for  exposure  rather  than  mastery.  We  find  a disturbingly  slow- 
moving  curriculum.  Much  of  the  content  taught  at  one  grade  is  repeated  in  the  next.  Like  TIMSS, 
we  find  teachers  not  covering  key  content  areas  on  which  student  achievement  has  been  shown  to  be 
weak.  For  example,  we  find  little  teacher  emphasis  on  geometry  and  measurement  in  mathematics. 

In  contrast,  we  find  teachers  emphasizing  basic  topics,  such  as  number  sense  and  calculation,  to  a 
greater  extent  than  is  found  on  NAEP.  The  emphasis  on  basic  topics  is  especially  true  in  high- 
poverty  schools. 

One  positive  finding  is  that  teaching  practices  appear  to  be  moving  toward  a better  balance 
across  performance  goals  for  students.  In  earlier  surveys  of  teachers'  practices,  a greater  emphasis 
was  found  to  be  given  to  student  memorization  of  facts  and  mastery  of  low-level  algorithmic  skills, 
while  less  emphasis  was  given  to  students'  ability  to  apply  their  knowledge  and  to  communicate  with 
others  (Porter  et  al.,  1993).  Our  longitudinal  study  baseline  data  show  a much  more  balanced 
emphasis  in  instruction  across  these  varying  performance  goals. 
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While  our  survey  findings  concerning  content  are  supported  in  part  by  our  observations  of 
teachers,  they  also  occasionally  are  challenged.  When  our  observations  reveal  differences  between 
what  we  saw  and  what  teachers  report,  the  differences  tend  to  be  in  the  direction  of  teachers’  over- 
reporting solving  novel  problems  and  applying  mathematics  and  science  to  real-world  situations. 

Still,  our  observations  are  largely  supportive  of  the  descriptions  of  classroom  practices  provided  by 
teachers  in  response  to  the  survey  questions.  As  reported  by  Cohen  (1990),  teachers  appear  to  be 
getting  the  message  from  national  professional  content  standards  that  better  balance  needs  to  occur 
across  performance  goals,  and  they  are  trying  to  move  in  that  direction.  In  at  least  some  cases, 
however,  their  practice  is  not  yet  there. 

Our  findings  about  teacher  classroom  practices  are  important  in  two  respects.  First,  the 
inadequacies  identified  in  current  instruction  suggest  important  targets  for  Eisenhower  and  other 
professional  development.  Second,  the  weaknesses  in  instruction  we  have  identified  represent  areas 
in  which  we  hope  to  track  improvement  over  time,  using  our  longitudinal  study,  and  to  relate 
whatever  improvement  is  observed  to  characteristics  of  professional  development  experienced  by  the 
teachers. 


Teachers’  Experiences  in  Eisenhower- Assisted  Activities 


We  examined  teachers'  experiences  in  Eisenhower-assisted  activities  in  three  ways.  First,  we 
assessed  the  effectiveness  of  Eisenhower-assisted  activities  by  asking  teachers  to  report  on  the  extent 
to  which  participation  enhanced  their  knowledge  and  skills.  Next,  we  examined  the  quality  of 
Eisenhower-assisted  activities,  be  assessing  the  extent  to  which  supported  activities  share  features  of 
quality  identified  in  the  literature  on  professional  development.  Finally,  we  used  our  data  to  assess 
the  strength  of  the  relationship  between  features  of  the  activities  in  which  teachers  participated  and 
teachers'  self-reported  outcomes. 

Effectiveness  of  Eisenhower- Assisted  Professional  Development  Activities 

To  measure  the  effectiveness  of  Eisenhower-assisted  professional  development  activities,  we 
asked  teachers  to  report  on  the  extent  to  which  participation  enhanced  their  knowledge  and  skills  in 
six  domains:  (1)  in-depth  knowledge  of  mathematics/science;  (2)  curriculum;  (3)  instructional 
methods;  (4)  approaches  to  assessment;  (5)  use  of  technology;  and  (6)  approaches  to  diversity.3 

When  asked  directly,  many  teachers  in  SAHE-grantee  activities  and  somewhat  fewer  teachers 
in  district  activities  report  that  participation  in  Eisenhower-supported  professional  development  has 
led  to  enhanced  knowledge  and  skills  and  changes  in  their  classroom  teaching  practice.  For 
enhanced  knowledge  and  skills,  we  asked  teachers  questions  about  six  domains,  ranging  from 
enhanced  knowledge  of  mathematics  and  science,  to  instructional  methods,  to  approaches  to 
diversity.  For  example,  48  percent  of  teachers  in  district  activities  and  68  percent  of  teachers  in 
SAHE-grantee  activities  report  enhanced  in-depth  knowledge  of  mathematics  or  science;  and  63 
percent  of  teachers  in  district  activities  and  79  percent  of  teachers  in  SAHE-grantee  activities  report 
enhanced  knowledge  of  instructional  methods.  Percentages  are  lower  for  use  of  technology  and 


3 The  term  "Eisenhower-assisted  activities"  reflects  the  fact  that  district  Eisenhower  funds  can  support  some  or  all  of 
the  cost  of  professional  development  activities. 


approaches  to  diversity.  Similarly,  we  asked  about  improvement  in  classroom  practice,  and  the 
percentages  were  parallel.  Once  again,  there  is  considerable  variability  among  activities  in  reported 
change  in  practice,  and  teachers  in  SAHE-grantee  activities  report  somewhat  more  change  in  practice 
than  do  teachers  in  district  activities.  Overall,  our  data  on  knowledge  and  skills  and  change  in 
teaching  practice  suggest  that  teachers  believe  they  are  benefiting  from  their  participation  in 
Eisenhower-assisted  professional  development. 

A comparison  of  our  data  for  SAHE-grantee  activities  with  the  results  obtained  for  34 
exemplary  summer  institutes  in  mathematics  and  science,  supported  by  the  National  Science 
Foundation,  the  Department  of  Education,  and  other  agencies,  indicates  that  teachers  participating  in 
SAHE-grantee  activities  report  enhancement  of  knowledge  and  skills  in  mathematics  and  science 
content  roughly  comparable  to  the  results  for  the  34  exemplary  activities.4  SAHE-grantee  activities 
meet  benchmarks  for  reported  teacher  outcomes  set  by  other  exemplary  professional  development 
programs.  Teachers  participating  in  district  Eisenhower  activities,  however,  show  weaker  results 
than  do  teachers  in  the  34  exemplary  activities. 

The  Quality  of  Eisenhower-assisted  Activities 

Our  analysis  of  the  quality  of  Eisenhower-assisted  professional  development  activities,  which 
appears  in  Chapter  3,  is  grounded  in  the  recent  literature  on  the  characteristics  of  professional 
development.  Over  the  past  decade,  a considerable  literature  has  emerged  on  professional 
development,  teacher  learning,  and  teacher  change  (Corcoran,  1995;  Darling-Hammond,  1995; 
Hargreaves  & Fullan,  1992;  Hiebert,  1999;  Lieberman,  1996;  Little,  1993;  Loucks-Horsley  et  al., 
1998;  Richardson,  1994;  Sparks  & Loucks-Horsley  1989;  Stiles,  Loucks-Horsley,  & Hewson,  1996). 
The  research  literature  contains  a mix  of  large-  and  smalfscale  studies,  including  intensive  case 
studies  of  classroom  teaching,  evaluations  of  programs  designed  to  improve  teaching  and  learning, 
and  surveys  of  teachers  about  their  preservice  and  in-service  preparation  and  in-service  professional 
development  experiences.  In  addition,  there  is  a large  literature  describing  "best  practices"  in 
professional  development,  drawing  on  expert  experiences. 

Despite  the  size  of  the  literature,  however,  relatively  little  systematic  research  has  been 
conducted  on  the  effects  of  professional  development  on  improvements  in  teaching  or  on  student 
outcomes,  and  very  little  has  been  conducted  comparing  the  effects  of  alternative  forms  of 
professional  development.  The  research  that  has  been  conducted,  however,  along  with  the 
experience  of  expert  practitioners,  does  provide  some  preliminary  guidance  about  the  characteristics 
of  high-quality  professional  development  (See  Loucks-Horsley,  1998.)  In  particular,  several  recent 
studies  suggest  that  professional  development  that  focuses  on  specific  mathematics  and  science 
content  and  the  ways  students  learn  such  content  is  especially  helpful  (Cohen  and  Hill,  1998; 
Kennedy,  1998).  To  measure  the  quality  of  Eisenhower-assisted  activities,  we  integrated  and 
operationalized  the  ideas  in  the  literature  on  "best  practices"  in  professional  development. 


4 See  Carey,  N.,  & Frechtling,  J.  (1997,  March).  Best  practice  in  action:  Follow-up  survey  on  teacher  enhancement 
programs.  Arlington,  VA:  National  Science  Foundation.  Carey  and  Frechtling  indicate  that  44  percent  of 
participants  in  outstanding  teacher  development  programs  reported  that  the  programs  enhanced  their  knowledge  and 
understanding  of  science  content  to  “a  great  extent”  (value  of  5 on  their  5-point  scale).  If  we  isolate  the  percentage 
of  participants  in  SAHE-grantee  activities  who  reported  that  the  activity  enhanced  their  mathematics  or  science 
knowledge  “to  a great  extent”  (value  of  5 on  the  5-point  scale),  the  percentage  is  41  percent.  The  comparable 
percent  for  district  activities  is  24  percent. 
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As  we  reported  in  Chapter  3,  we  focus  on  three  “structural  features,"  or  characteristics  of  the 
structure  of  a professional  development  activity.  These  structural  features  include: 

♦ the  form  or  organization  of  the  activity — that  is,  whether  the  activity  is  organized  as  a 
reform  type,  such  as  a study  group,  teacher  network,  mentoring  relationship,  committee 
or  task  force,  internship,  individual  research  project,  or  teacher  research  center,  in 
contrast  to  a traditional  workshop  or  conference; 

♦ the  duration  of  the  activity,  including  the  total  number  of  contact  hours  that  participants 
are  expected  to  spend  in  the  activity,  as  well  as  the  span  of  time  over  which  the  activity 
takes  place;  and 

♦ the  degree  to  which  the  activity  emphasizes  the  collective  participation  of  groups  of 
teachers  from  the  same  school,  department,  or  grade  level,  as  opposed  to  the  participation 
of  individual  teachers  from  many  schools. 

In  addition  to  these  structural  features,  we  focus  on  three  “core  features”  or  characteristics  of 
the  substance  of  the  professional  development  experience  itself: 

♦ the  degree  to  which  the  activity  has  a content  focus — that  is,  the  degree  to  which  the 
activity  is  focused  on  improving  and  deepening  teachers’  content  knowledge  in 
mathematics  and  science; 

♦ the  extent  to  which  the  activity  offers  opportunities  for  active  learning  --  that  is, 
opportunities  for  teachers  to  become  actively  engaged  in  ‘.u‘7‘  meaningful  analysis  of 
teaching  and  learning,  for  example,  by  reviewing  student  work  or  obtaining  feedback  on 
their  teaching;  and 

♦ the  degree  to  which  the  activity  promotes  coherence  in  teachers’  professional 
development,  by  incorporating  experiences  that  are  consistent  with  teachers’  goals, 
aligned  with  state  standards  and  assessments,  and  encouraging  of  continuing  professional 
communication  among  teachers. 

We  used  these  structural  and  core  features  to  judge  the  extent  to  which  the  Eisenhower 
program  supports  high-quality  professional  development.  We  found  that  most  Eisenhower-assisted 
activities  are  traditional  in  form,  such  as  workshops,  courses,  or  conferences;  and  relatively  few 
Eisenhower-assisted  activities  are  reform  types  of  activities,  such  as  study  groups,  networks,  or 
mentoring  relationships.  About  79  percent  of  teachers  in  district  activities  are  in  traditional  types  of 
activities — primarily  workshops  and  conferences.  About  76  percent  of  teachers  in  SAHE-grantee 
activities  are  also  in  traditional  activities — primarily  workshops  and  college  courses.  In  contrast, 
approximately  22  percent  of  teachers  in  district  activities  and  26  percent  in  SAHE-grantee  activities 
are  in  reform  activities,  including  teacher  networks,  study  groups,  mentoring,  committees  and  task 
forces,  internships,  and  resource  centers. 

On  average,  SAHE-grantee  activities  are  of  longer  duration  than  district  activities.  The 
average  amount  of  time  teachers  in  district  activities  report  spending  in  Eisenhower-assisted 
activities  is  25  hours,  compared  to  51  hours  for  teachers  in  SAHE-grantee  activities.  However,  while 
district  activities  are  shorter  than  SAHE-grantee  activities,  the  average  length  of  district  activities,  in 


hours,  has  approximately  doubled  since  the  last  evaluation  was  conducted  in  1988-89. 5 In  addition,  a 
higher  percentage  of  SAHE-grantee  activities  than  district  activities  span  an  extended  period  of  time. 
For  example,  46  percent  of  teachers  in  SAHE-grantee  activities  are  in  activities  lasting  at  least  six 
months,  including  2 percent  lasting  more  than  one  year,  20  percent  lasting  10-12  months,  and  24 
percent  lasting  6-9  months.  Twenty  percent  of  teachers  in  district  activities  are  in  activities  lasting  at 
least  six  months. 

On  the  third  structural  feature,  collective  participation,  districts  outperform  SAHE  grantees. 
Although  relatively  few  district  or  SAHE-grantee  activities  emphasize  the  collective  participation  of 
teachers  from  the  same  department,  grade  level,  or  school,  district  activities  give  more  emphasis  to 
collective  participation  than  do  SAHE-grantee  activities.  Twenty  percent  of  teachers  in  district 
Eisenhower-assisted  activities  report  participating  with  other  teachers  in  their  department  or  grade 
level,  compared  to  7 percent  of  teachers  in  SAHE-grantee  activities;  and  19  percent  of  teachers  in 
district  activities  report  participating  with  all  teachers  in  their  school,  compared  to  1 1 percent  of 
teachers  in  SAHE-grantee  activities. 

On  the  last  three  dimensions  of  quality — content  focus,  active  learning,  and  coherence — we 
find  that  SAHE-grantee  professional  development  is  more  likely  to  place  an  emphasis  on  these 
features  than  district  professional  development. 

Two  thirds  of  teachers  participating  in  SAHE-supported  Eisenhower  activities  participate  in 
activities  that  place  a major  emphasis  on  content,  compared  to  fifty-one  percent  of  teachers  in  district 
Eisenhower-assisted  activities.  Similarly,  the  percentage  of  teachers  reporting  a major  emphasis  on 
each  of  1 8 separate  indicators  of  active  learning  is  higher  for  SAHE  grantees  than  for  districts,  but 
relatively  few  teachers  in  either  district  or  SAHE-grantee  activities  report  some  elements  of  active 
learning.  For  example,  only  5 percent  of  teachers  in  district  activities  and  16  percent  of  teachers  in 
SAHE-grantee  activities  report  that  their  teaching  was  observed  by  the  activity  leader. 

Teachers  in  both  district  and  SAHE-grantee  activities  report  that  the  activities  have  elements 
that  promote  coherence  with  other  aspects  of  their  professional  experiences.  For  example,  more  than 
three  quarters  of  teachers  in  both  district  and  SAHE-grantee  activities  report  that  their  activities  are 
aligned  with  state  and  district  standards.  Somewhat  more  teachers  in  SAHE-grantee  than  district 
activities,  however,  report  that  their  Eisenhower  activities  built  on  prior  professional  development 
(39  percent  compared  to  31  percent)  or  were  followed  up  with  later  activities  (70  percent  compared 
to  53  percent). 

Our  main  finding  is  that  there  is  enormous  variability  in  quality  among  Eisenhower-assisted 
activities.  This  is  not  surprising,  given  that  the  Eisenhower  program  operates  as  a funding  stream 
within  broad  guidelines.  A great  deal  of  discretion  is  left  to  districts  and  SAHE  grantees  to  decide  on 
the  nature  of  the  professional  development  that  they  will  provide,  though  SAHE  grantees  have  to 
design  programs  within  the  constraints  of  the  SAHE's  competition.  The  variability  in  quality 
indicates  that  some  districts  are  providing  the  same  kinds  of  high-quality  professional  development 
activities  that  SAHE  grantees  provide.  For  example,  although  many  district  activities  are  short,  2 
percent  of  teachers  in  district  activities  are  in  activities  spanning  more  than  one  year,  8 percent  are  in 
activities  lasting  10-12  months,  and  10  percent  are  in  activities  lasting  6-9  months.  That  some 


5 The  1988-99  evaluation  collected  data  on  duration  from  districts  rather  than  teachers,  so  a comparison  of  results 
from  the  1988-89  and  the  current  evaluation  should  be  interpreted  as  providing  an  indication  of  the  general 
magnitude  of  the  change  rather  than  a precise  numerical  estimate  (see  Knapp  et  al.,  1991). 
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districts  provide  activities  of  extended  duration,  with  collective  participation,  a major  focus  on 
content  knowledge,  a major  emphasis  on  active  learning,  and  coherence  with  teachers'  other 
experiences,  represents  an  “existence  proof’  that  it  is  possible  for  districts  to  provide  such  activities. 

We  do  not  know,  however,  whether  all  districts  possess  the  organizational  capacity  and 
staffing  to  provide  quality  professional  development.  Districts  lacking  capacity  may  well  be  able  to 
purchase  high-quality  professional  development  (e.g.,  from  institutions  of  higher  education)  if  they 
are  aware  of  and  insist  upon  the  dimensions  of  quality  we  have  identified. 

Finally,  the  average  differences  in  quality  we  observe  between  district  and  SAHE-grantee 
activities  are  associated  with  parallel  differences  in  cost.  Thus,  we  conclude  that  high-quality 
professional  development  is  more  expensive  than  lower-quality  professional  development.  SAHE 
grantees  spend  over  twice  as  much  per  teacher  participant  as  do  districts.  We  estimated  that  SAHE 
grantees  spend  about  $512  per  participation,  in  comparison  to  $185  per  participation  for  districts.6 
High-quality  professional  development  is  more  expensive. 

Why  the  SAHE-grantee  activities  are  of  higher  quality,  on  average,  than  the  district  activities 
is  not  completely  clear.  One  possible  explanation  is  that  districts  receive  a formula  allotment  while 
the  SAHE  grantees  must  compete  for  their  award.  Both  the  competition  and  the  SAHE  guidelines  for 
the  competition  may  push  in  directions  of  higher  quality.7  In  addition,  IHE/NPOs  may,  on  average, 
possess  more  state-of-the-art  knowledge  about  the  attributes  of  high-quality  professional 
development  than  districts  do,  and  they  may  have  greater  capacity  to  deliver  professional 
development  with  these  attributes.  Finally,  as  noted,  high-quality  professional  development  is 
considerably  more  expensive  per  participation.  Undoubtedly,  districts  feel  a responsibility  to  provide 
professional  development  to  all  of  their  teachers.  This  may  push  them  in  the  direction  of  professional 
development  with  lower  costs  per  participation. 

Features  of  Professional  Development  that  Promote  Improvements 
in  Teaching 

We  used  our  national  probability  sample  of  teachers  who  participated  in  Eisenhower-assisted 
professional  development  to  estimate  the  strength  of  the  relationships  among  features  of  the 
professional  development  and  self-reported  teacher  outcomes — enhanced  knowledge  and  skills  and 
changed  teaching  practice.  Our  analyses,  reported  in  Chapter  3 (Exhibit  3.21),  estimate  standardized 
path  coefficients  while  controlling  for  the  subject  area  of  the  professional  development  experience 
(mathematics  or  science);  school  poverty,  percent  minority,  and  grade  level;  and  teacher  gender, 
certification,  and  years  of  experience. 

We  find  that  activities  with  more  positive  structural  features  tend  to  provide  professional 
development  experiences  with  more  positive  core  features,  which  in  turn  tend  to  produce  more 
positive  teacher  outcomes.  In  particular,  activities  of  longer  duration,  both  in  time  span  and  in 
contact  hours,  tend  to  place  more  emphasis  on  content  than  shorter  activities,  provide  more 
opportunities  for  active  learning,  and  provide  more  coherent  professional  development.  Activities 
with  more  collective  participation  also  tend  to  provide  more  opportunities  for  active  learning  and 


6 A “participation”  is  a teacher  participant  in  an  Eisenhower-assisted  activity.  Teachers  who  participate  in  more  than 
one  activity  are  counted  separately  for  each  activity  in  which  they  participate.  The  dollar  per  participation  figure  for 
districts  includes  federal  Eisenhower  dollars  only,  and  does  not  count  the  33  percent  matching  requirement. 

7 We  were  not  able  to  conduct  a systematic  analysis  of  SAHE  competitions. 


7-9 


offer  more  coherent  professional  development.  The  three  core  features,  in  turn,  all  have  independent 
effects  on  teachers’  reports  of  enhanced  knowledge  and  skills.  Teachers  who  participate  in  activities 
that  place  a major  emphasis  on  mathematics  and  science  content,  for  example,  are  substantially  more 
likely  to  report  enhanced  knowledge  and  skills  than  teachers  in  activities  that  do  not  emphasize 
content.  (For  a more  complete  description,  see  Chapter  3.) 

Thus,  our  results  expand  the  existing  literature  by  providing  empirical  support  for  the 
contention  that  professional  development  that  emphasizes  content  knowledge,  active  learning,  and 
coherence  leads  to  enhanced  teacher  knowledge  and  skill  and  change  in  teacher  practice.  In  addition, 
our  results  highlight  the  importance  of  professional  development  that  engages  teachers  for  an 
extended  number  of  contact  hours,  and  that  spans  several  months,  as  opposed  to  being  concentrated 
in  a few  days  or  weeks. 

Our  results  also  support  the  literature  in  indicating  that  reform  types  of  professional 
development  tend  to  produce  more  positive  outcomes  than  traditional  types,  but  the  effect  is  smaller 
than  we  had  anticipated.  In  our  analyses,  we  find  effects  of  the  distinction  between  traditional  and 
reform  activities,  and  they  are  in  the  predicted  direction,  but  they  are  generally  not  direct  effects  on 
teacher  outcomes.  Rather,  the  effect  of  reform  versus  traditional  professional  development  activities 
operates  indirectly  through  the  structural  and  core  features  identified  above.  That  is,  reform  activities 
tend  to  produce  better  outcomes  primarily  because  they  tend  to  be  of  longer  duration.  Traditional 
and  reform  activities  of  the  same  duration  tend  to  have  the  same  effects  on  reported  outcomes. 
Further,  we  find  a large  amount  of  variability  within  both  traditional  and  reform-type  activities. 
Traditional  activities  sometimes  have  positive  characteristics,  such  as  opportunities  for  active 
learning,  and  reform  activities  sometimes  have  negative  characteristics,  such  as  limited  duration. 

Further,  our  analyses  show  that  the  difference  between  districts  and  SAHE  grantees,  in  terms 
of  enhanced  teacher  knowledge  and  skill  and  greater  change  in  teacher  practice,  is  explained  almost 
entirely  by  the  SAHE  grantees'  placing  a greater  emphasis  upon  duration,  subject-matter  content, 
active  learning,  and  coherence.  In  short,  if  districts  placed  a greater  emphasis  upon  these 
characteristics,  we  would  hypothesize  that  teachers  would  report  their  knowledge  and  skill  enhanced 
to  the  same  extent  as  teachers  in  SAHE-grantee  activities,  and  that  they  would  be  just  as  likely  to 
report  changing  their  teaching  practice. 


District  Management  and  Operation  of 
Eisenhower- Assisted  Activities 


We  now  turn  from  describing  teachers’  experiences  in  Eisenhower-assisted  activities  to  a 
description  of  the  ways  that  districts  manage  and  operate  their  Eisenhower  programs.  Each  district 
receiving  Eisenhower  funds  generally  uses  the  funds  to  support  a collection  of  professional 
development  activities.  The  collection  of  professional  development  activities  that  a district  supports 
with  Eisenhower  funds  can  be  viewed  in  its  entirety  as  its  “portfolio”  of  Eisenhower-assisted 
professional  development  activities.  In  our  analysis,  we  examine  the  factors  that  influence  the 
quality  of  the  portfolios  of  professional  development  activities  that  districts  offer  and  the  extent  to 
which  districts  engage  in  efforts  to  target  teachers  of  high-need  students. 
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The  measures  we  use  to  characterize  the  quality  of  a district’s  portfolio  of  Eisenhower- 
assisted  activities  are:  (1)  the  percent  of  participations  in  reform  types  of  activities,  (2)  the  average 
span  of  time  of  activities  (i.e.,  number  of  days,  weeks,  or  months),  (3)  the  number  of  opportunities 
for  active  learning  in  in-district  workshops  and  institutes,  and  (4)  the  degree  of  collective 

O 

participation  in  in-district  workshops  and  institutes. 

In  assessing  district  management  and  operations,  we  focus  on  the  role  of  several  provisions 
emphasized  in  the  legislation:  (1)  the  coordination  ( co-funding ) of  Eisenhower- assisted  activities 
with  other  sources  of  funding  for  professional  development;  (2)  the  alignment  of  Eisenhower-assisted 
activities  with  state  and  district  standards  and  assessments;  (3)  the  participation  of  teachers  and 
school-level  staff  in  planning  Eisenhower- assisted  activities;  and  (4)  the  use  of  a process  of 
continuous  improvement , including  monitoring  progress  against  measurable  objectives  and 
performance  indicators. 

The  Relationship  of  District  Management  and  Features  of  Professional 
Development 

In  Chapter  5,  we  report  the  results  of  a path  analysis  of  the  relationship  between  these 
provisions  of  the  legislation  and  the  characteristics  of  quality  professional  development  that  districts 
provide.  The  path  analysis  is  based  on  a national  probability  sample.  Our  path  analysis  of  district 
professional  development  portfolios  (Exhibit  5.11)  controlled  for  district  size,  poverty  level, 
consortium,  and  cluster  status.  We  began  our  analysis  of  districts  by  examining  co-funding  (one 
aspect  of  coordination)  and  alignment.  Consistent  with  the  literature  on  systemic  reform  (e.g., 
Fuhrman,  1993;  Smith  & O'Day,  1991),  we  find  that  co-funding  Eisenhower  professional 
development  activities  with  other  professional  development  initiatives  has  important  direct  and 
indirect  effects  on  the  quality  of  the  Eisenhower-assisted  professional  development  offered  by 
districts.  For  example,  districts  that  engage  in  more  co-funding  of  Eisenhower  activities  with  other 
programs  tend  to  support  a greater  proportion  of  reform  activities  than  districts  that  engage  in  less  co- 
funding, and  they  tend  to  provide  more  opportunities  for  collective  participation.  In  addition, 
districts  that  engage  in  more  co-funding  tend  to  engage  in  more  extensive  continuous  improvement 
efforts,  and  they  tend  to  involve  teachers  more  widely  in  planning,  both  of  which  are  related  to 
increased  opportunities  for  active  learning.  Alignment  of  professional  development  with  state  and 
district  standards  and  assessments  also  has  positive  effects.  Districts  that  align  professional 
development  with  standards  and  assessments  are  more  likely  to  offer  reform  types  of  activities.  In 
addition,  these  districts  are  more  likely  than  others  to  engage  in  continuous  improvement,  which  is 
related  to  increased  opportunities  for  active  learning.  These  results  (along  with  others  reported  in 
Chapter  5)  lead  us  to  conclude  that  the  intentions  of  the  program  concerning  strategies  such  as  co- 
funding, alignment,  continuous  improvement,  and  teacher  involvement  in  planning  are  appropriate, 
in  the  sense  that  they  appear  to  lead  to  higher-quality  professional  development. 

Thus,  through  analyses  of  our  data,  we  find  support  for  the  appropriateness  of  the  key 
provisions  of  the  Eisenhower  legislation.  Generally,  these  provisions  tend  to  encourage  districts  to 


8 The  data  from  our  national  sample  of  teachers  show  that  each  of  these  dimensions  is  related,  either  indirectly  or 
directly,  to  improvements  in  teachers’  knowledge  and  skills  and  changes  in  teaching  practice;  thus,  we  consider  each 
of  these  dimensions  as  an  indicator  of  high-quality  professional  development,  whether  it  has  a direct  effect  on 
teacher  outcomes,  or  operates  indirectly  (e.g.,  a reform  approach  affects  teacher  outcomes  indirectly  through  its 
effect  on  duration). 
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offer  professional  development  with  effective  structural  and  core  characteristics,  as  identified  in  the 
literature  and  supported  by  our  evaluation. 

Variation  Among  Districts  in  Management  and  Operations 


Given  that  co-funding,  alignment,  teacher  participation  in  planning,  continuous  improvement, 
and  targeting  are  intended  features  of  the  program,  and  given  that  our  evidence  indicates  that  these 
strategies  generally  lead  toward  higher-quality  professional  development,  we  ask  to  what  extent  each 
is  being  implemented  (Chapter  4). 

Co-funding  and  working  closely  with  other  federally  supported  professional  development 
programs  (where  the  programs  are  available)  occur  most  often  with  professional  development 
programs  that  have  a mathematics  and/or  science  focus  similar  to  that  of  the  Eisenhower  program. 
Thus,  working  closely  with  other  programs  and  co-funding  is  more  likely  between  the  Eisenhower 
program  and  National  Science  Foundation  programs  than  it  is  with  other  Department  of  Education 
programs.  For  example,  among  teachers  in  districts  with  an  NSF-funded  Urban  Systemic  Initiative 
(USI)  that  supports  professional  development,  over  80  percent  are  in  districts  where  the  Eisenhower 
project  co-funds  with  the  USI.  Among  teachers  in  districts  with  Title  I,  Part  A funds,  50  percent  are 
in  districts  where  the  Eisenhower  project  co-funds  with  Title  I. 

Most  districts  report  being  engaged  in  efforts  to  align  Eisenhower-assisted  professional 
development  with  standards  and  assessments.  Alignment  is  more  likely  for  state  standards  and 
assessments  than  it  is  for  district  standards  and  assessments  and  more  likely  for  mathematics  than 
science.  In  addition,  alignment  is,  however,  more  likely  for  standards  than  for  assessments.  This 
may  reflect  the  fact  that  the  ESEA  requirement  of  aligned  assessments  had  not  yet  gone  into  effect; 
alternatively,  these  results  may  reflect  that  districts  first  establish  standards,  and  then  seek  to  revise 
their  assessments  to  be  aligned  with  the  newly  developed  standards. 

The  program  intends  that  teachers  participate  in  planning  Eisenhower-assisted  activities. 

Our  results  indicate  that  99  percent  of  teachers  are  in  districts  that  report  involving  teachers  in 
planning  professional  development.  Thus,  we  conclude  that  this  provision  of  the  law  is  relatively 
well-implemented.  Still,  our  case-study  data  reveal  that  the  Eisenhower  legislation's  80-20  rule, 
calling  for  at  least  80  percent  of  funds  to  be  used  "in  a manner  that  is  determined  by  such  teachers 
and  staff'  and  "to  the  extent  practicable,  takes  place  at  the  individual  school  site,"  is  not  well- 
understood  and  is  sometimes  not  even  known  to  exist.  We  conclude  that  this  provision  of  the  law 
could  be  clarified  and  given  more  emphasis  in  the  information  provided  to  projects  from  the 
Department  of  Education  and  from  the  states. 

Our  data  suggest  that  the  use  of  performance  indicators  to  guide  the  continuous  improvement 
process  is  not  yet  widespread  in  districts  that  receive  Eisenhower  funds.  Fewer  than  one  in  five 
teachers  are  in  districts  that  collect  data  on  performance  indicators  established  to  guide  district 
professional  development  efforts.  One-third  of  the  teachers  are  in  districts  that  have  developed  such 
indicators.  The  majority  of  teachers  are  in  districts  whose  Eisenhower  coordinators  are  not  aware 
that  their  state  has  performance  indicators.  Clearly,  the  development  of  state  and  district 
performance  indicators  is  a goal  of  the  program  that  is  not  generally  being  implemented,  at  least  not 
yet.  There  are  other  ways  in  which  districts  show  some  commitment  to  continuous  improvement. 
Needs  assessments  through  teacher  surveys  and  informal  conversations  are  common.  Nearly  every 
district  reports  evaluating  their  professional  development  activities.  These  evaluations,  however,  are 
frequently  based  on  teacher  satisfaction  surveys  or  participation  counts.  It  is  less  common  to  use 
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observations  of  teachers’  subsequent  classroom  practice  to  evaluate  the  effectiveness  of  Eisenhower 
professional  development. 

Districts  report  a strong  emphasis  on  recruiting  teachers  from  high-poverty,  low-achievement 
schools,  and  slightly  less  emphasis  on  recruiting  teachers  from  Title  I schools.  Despite  these  reported 
efforts  at  targeting,  and  even  though  districts  with  larger  numbers  of  high-poverty  students  receive 
greater  funding,  teachers  from  high-poverty  schools  are  only  slightly  more  likely  than  others  to 
participate  in  Eisenhower-funded  activities.  Apparently,  current  targeting  strategies  have  limited 
effectiveness.  Perhaps  these  results  are  explained,  in  part,  by  the  fact  that  most  participants  in 
Eisenhower- assisted  activities  are  volunteers  and  teachers  from  high-poverty  schools  are  no  more  or 
less  likely  than  other  teachers  to  volunteer  for  professional  development.  It  is  possible  that  districts 
do  not  have  adequate  strategies  to  shape  the  incentives  and  constraints  that  determine  which  teachers 
volunteer. 

Differences  in  Management  and  Operation  of  Eisenhower-assisted  Activities 
by  District  Poverty  and  Size 

Throughout  our  analyses  of  district  data,  we  tested  to  see  where  the  management  and 
operation  of  Eisenhower- assisted  professional  development  differ  significantly  according  to  the 
district  poverty  level  or  the  size  of  the  district.  All  of  our  analyses  simultaneously  controlled  for  size 
and  poverty,  so  any  effects  are  independent  of  one  another.  We  also  tested  for  the  interaction 
between  these  two  variables.  Of  the  two  variables,  district  size  is  much  more  frequently  related  to 
the  nature  of  the  Eisenhower  professional  development  provided  than  is  poverty.  Generally,  larger 
districts  are  more  likely  to  provide  high-quality  professional  development  than  are  smaller  districts. 
This  is  true  for  features  of  the  activities  provided — such  as  active  learning,  collective  participation, 
and  the  span  of  time  over  which  the  activities  extend — as  well  as  district  management  strategies, 
including  alignment  with  standards  and  assessments,  frequency  of  co-funded  projects,  and  a 
commitment  to  continuous  improvement.  Also,  when  several  small  districts  form  consortia  to  deliver 
Eisenhower  professional  development,  the  quality  is  higher  in  ways  that  parallel  that  for  large 
districts.  We  speculate  that  larger  districts  and  consortia  have  a better  infrastructure  and  greater 
capacity  to  provide  high-quality  professional  development.  Higher-poverty  districts  also  are  more 
likely  to  co-fund  Eisenhower  professional  development  with  other  professional  development,  and 
they  are  more  likely  to  be  committed  to  continuous  improvement.  Large  districts  and  high-poverty 
districts  have  a greater  variety  of  funding  sources  in  addition  to  Eisenhower,  increasing  opportunities 
for  co-funding,  and  perhaps  creating  a complexity  that  demands  a commitment  to  continuous 
improvement. 


SAHE-Grantee  Management  and  Operation  of 
Eisenhower-Assited  Activities 


SAHE  grantees  are  subject  to  the  same  provisions  for  alignment,  continuous  improvement, 
and  targeting  that  districts  must  follow.  However,  SAHE  grantees  also  are  subject  to  some  additional 
requirements  or  guidelines.  In  particular,  the  1994  reauthorization  emphasizes  the  importance  of 
coordination  between  SAHE  grantees  and  districts  in  planning  and  providing  professional 
development.  Each  SAHE  grantee  is  required  to  enter  into  an  agreement  with  one  or  more  districts 
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for  the  provision  of  professional  development.  Further,  SAHE-grantee  projects  are  shaped  by  the 
priorities  and  guidelines  that  the  SAHE  sets  in  structuring  its  competition  for  the  Eisenhower 
awards.9 

The  Relationship  of  SAHE-grantee  Management  to  Features  of  Professional 
Development 

In  Chapter  6,  we  report  the  results  of  our  analyses  of  the  relationship  between  intended 
program  strategies,  such  as  alignment,  coordination,  and  continuous  improvement,  and  the  quality  of 
the  professional  development  that  SAHE  grantees  provide.  The  path  analysis  is  based  on  a national 
probability  sample. 

As  reported  in  Chapter  6,  Exhibit  6.26,  coordination  with  districts  (e.g.,  use  of  feedback 
mechanisms,  support  and  extension  of  district  activities,  co-funding  and  district  involvement  in 
planning,  implementing  and  monitoring)  is  positively  associated  with  the  quality  of  the  professional 
development  provided.  For  example,  SAHE  grantees  that  coordinate  more  extensively  with  districts 
provide  professional  development  with  a larger  number  of  contact  hours.  Coordination  also  has  a 
positive  effect  on  the  use  of  strategies  for  continuous  improvement  which,  in  turn,  increases  the 
active  learning.  In  addition,  coordination  has  an  indirect  effect  upon  targeting,  through  its  effect 
upon  continuous  improvement.  These  results  for  coordination  parallel  those  found  for  the  district 
sample. 

Variation  Among  SAHE  Grantees  in  Management  and  Operations 

We  have  discussed  the  legislative  requirements  for  alignment,  coordination,  co-funding, 
continuous  improvement,  and  targeting,  and  their  role  in  shaping  effective  professional  development 
in  SAHE-grantee  projects;  we  now  discuss  the  extent  to  which  SAHE-grantee  projects  are 
implementing  these  management  strategies  (Chapter  6). 

Nearly  all  SAHE-grantee  projects  are  aligned  with  state  standards,  and  most  are  aligned  with 
state  assessments  and  district  standards,  but  less  than  half  are  aligned  with  district  assessments. 
SAHE  grantees  work  closely  with  districts  in  several  ways,  such  as  communicating  with  district  staff 
and  using  district  needs  assessments,  but  report  low  levels  of  other  key  components  of  coordination, 
such  as  co-funding  and  working  with  the  Eisenhower  coordinator.  Also,  SAHE-grantee  projects  tend 
to  be  much  more  affected  by  state  than  district  indicators.  However,  SAHE  grantees  report 
moderately  high  levels  of  engagement  in  most  other  continuous  improvement  efforts,  such  as 
conducting  needs  assessments  and  evaluations.  These  findings  suggest  that,  although  the  average 
quality  of  SAHE-grantee  activities  is  relatively  high,  it  could  be  further  improved  by  strengthening 
the  coordination  between  SAHE  grantees  and  districts  and  giving  more  emphasis  to  the  use  of  district 
indicators. 

In  addition,  although  most  SAHE-grantee  project  directors  report  targeting  teachers  of  low 
achieving,  high-poverty  students  and  teachers  in  Title  I schools,  these  teachers  are  much  less  likely 
than  others  to  participate  in  SAHE-grantee  activities. 

9 Relative  to  SEAs,  SAHEs  have  a smaller  number  of  grantees,  and  thus  may  be  able  to  monitor  their  grantees’ 
projects  to  help  ensure  the  implementation  of  quality  activities.  However,  we  did  not  examine  the  SAHE’s 
monitoring  role. 
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indicate  that  larger  districts  have  greater  capacity  and,  to  some  extent,  so  do  high-poverty  districts. 
This  larger  capacity  may  be  explained,  in  part,  by  their  larger  district  staff  and,  in  part,  by  their 
greater  Eisenhower  funds.  Consortia  that  tie  together  several  small  districts  into  one  unit  for 
providing  Eisenhower  professional  development  also  seem,  on  average,  more  effective  than  small 
districts.  The  issue  of  district  capacity  is  one  that  should  be  closely  monitored  in  future  research. 

Sixth,  districts  could  increase  the  quality  of  the  professional  development  they  provide  by 
focusing  their  Eisenhower  money  on  a small  number  of  teachers,  rather  than  spreading  it  across  a 
large  number  of  teachers.  Not  surprisingly,  high-quality  professional  development  costs  more  per 
participant  than  does  lower  quality  professional  development.  Districts  may  feel  a greater 
responsibility  to  reach  a large  number  of  teachers  than  do  SAHE  grantees,  and  this  is  reflected  in  the 
cost  per  participant.  The  question  is,  should  districts  continue  to  spread  the  money  from  the 
Eisenhower  program  across  as  many  teachers  as  possible?  Or,  should  they  focus  the  money  on  a 
small  number  of  teachers,  so  that  they  can  provide  higher  quality,  more  influential  professional 
development?  Our  results  suggest  the  money  should  be  focused.  This  recommendation  also  interacts 
with  the  finding  about  targeting.  More  effective  targeting  might,  at  the  same  time,  provide  a 
rationale  for  more  focused  expenditures. 

Seventh,  one  reason  that  SAHE-grantee  professional  development  is,  on  average,  of  higher 
quality  than  district  professional  development  may  be  that  SAHE  grantees  have  to  compete  for  funds. 
We  do  not  know  as  much  about  these  SAHE-sponsored  competitions  as  we  would  like.  We  find  that 
the  majority  of  SAHE  grantees  have  been  receiving  Eisenhower  support  for  several  years.  Still,  in  all 
cases,  IHE/NPOs  interested  in  Eisenhower  support  must  develop  a proposal  and  have  it  judged 
worthy  of  funding.  In  contrast,  districts  receive  money  from  the  state  educational  agency  on  a 
formula  basis,  with  no  competition.  Perhaps  having  districts  compete  for  funds  would  push  them 
toward  higher  quality  professional  development. 

Eighth,  there  is  considerable  evidence  that,  on  average,  education  departments  in  institutions 
of  higher  education  provide  higher  quality  professional  development  than  do  mathematics  and 
science  departments.  The  one  exception  is  that  mathematics/science  departments  in  non- 
research/doctoral institutions  focus  more  on  content  knowledge  than  do  education  departments.  We 
do  not  have  empirical  data  on  interdepartmental  collaboration,  but,  in  our  case  studies,  we  did  not  see 
much  evidence  of  these  two  types  of  departments  collaborating  and  combining  their  expertise  to 
provide  high-quality  professional  development.  Lack  of  collaboration  between  education  and 
mathematics/science  departments  is  a well-recognized  problem  in  preservice  teacher  education  and 
may  be  an  issue  for  in-service  education,  as  well. 

Ninth,  SAHE-grantee  projects  should  be  engaging  in  higher  levels  of  coordination  with 
districts,  as  called  for  in  the  legislation.  The  evaluation  shows  that  such  coordination  is  related  to  the 
provision  of  high-quality  professional  development.  Therefore,  more  attention  might  be  paid  to 
supporting  and  developing  opportunities  for  SAHE  grantees  to  coordinate  and  work  with  districts  in 
mutually  beneficial  ways — ways  that  allow  grantees  to  exercise  their  expertise  in  developing 
professional  development  projects  while  benefiting  from  district  expertise  in  serving  the  needs  of 
their  teachers  and  students. 
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Finally,  the  evaluation  supports  the  importance  of  programs  that  fund  professional 
development  activities  within  specified  subject  areas.  During  the  past  15  years,  the  Eisenhower 
Professional  Development  Program  has  provided  continuous  support  for  professional  development 
activities  for  mathematics  and  science.  This  evaluation  highlights  the  importance  of  the  content 
focus  of  professional  development  activities  and  the  role  that  the  Eisenhower  program  has  played  in 
building  capacity  in  these  two  subjects  in  school  districts.  Generic  professional  development  that 
focuses  on  teaching  techniques  without  a content  focus  does  not  appear  to  be  effective.  If  the 
Congress  is  considering  expanding  the  program,  it  should  consider  creating  analogous  programs  in 
other  academic  subject  areas,  rather  than  eliminating  the  content  focus  on  mathematics  and  science. 


333 


er|c 


, 7-18 


Differences  in  SAHE-grantee  Management  and  Operation  by  Institution  Type 
and  Departmental  Affiliation 


In  our  analysis  of  SAHE  grantees,  we  compared  activities  housed  in  education, 
mathematics/science,  and  other  departments,  and  research/doctoral  and  other  types  of  institutions  of 
higher  education.  Our  results  indicate  that,  on  average,  projects  housed  in  education  departments  are 
stronger  in  important  ways  than  are  projects  in  mathematics/science  departments.  Education 
departments  sponsor  professional  development  activities  that  last  more  than  twice  the  number  of 
contact  hours  and  span  a longer  time  period.  They  engage  in  more  types  of  coordination  with  the 
districts  from  which  they  draw  their  teachers,  and  they  place  more  emphasis  on  continuous 
improvement  than  do  mathematics/science  departments.  Education  department  projects  are 
especially  strong  when  housed  in  research/doctoral  universities,  where  they  are  more  likely  to  put  a 
high  focus  on  content  and  provide  numerous  opportunities  for  active  learning. 

These  findings  of  the  superiority  of  professional  development  provided  through  education 
departments,  in  contrast  with  mathematics/science  departments,  may  be  due  to  the  fact  that  education 
departments  have  greater  expertise  in  student  and  teacher  learning  of  subject  matter,  as  well  as 
experience  in  coordinating  with  practitioners.  Faculty  in  these  departments  are  the  main  contributors 
to  the  literature  on  effective  professional  development  in  mathematics  and  science.  Projects  in 
mathematics/science  departments,  on  the  other  hand,  have  a strong  content  focus.  Others  have  noted 
that,  in  the  training  of  teachers,  quality  might  be  enhanced  by  better  collaboration  between  educators 
and  scientists.  Apparently,  the  same  can  be  said  for  continuing  teacher  education  in  the  form  of 
Eisenhower  professional  development.  Each  type  of  department  brings  unique  strengths  to  designing 
and  delivering  professional  development  that  could  be  complementary  in  meaningful  collaboration. 


These  findings  of  the  National  Evaluation  of  the  Eisenhower  Professional  Development 
Program  have  a number  of  lessons  for  professional  development  generally  and  for  future  Eisenhower 
legislation  and  program  operations  specifically. 

First,  the  program  should  seek  ways  to  encourage  the  use  of  all  features  of  professional 
development  that  are  related  to  teacher  knowledge  and  skills,  and  change  in  practice.  The  program 
should  continue  emphasizing  "sustained  and  intensive"  professional  development  activities.  More 
specifically,  this  evaluation  suggests  that  professional  development  should  focus  on  deepening 
teachers'  content  knowledge  and  knowledge  of  how  students  learn  particular  content,  on  providing 
opportunities  for  active  learning,  and  on  encouraging  coherence  in  teachers'  professional 
development  experiences.  Eisenhower  professional  development  should  pursue  these  goals  using 
activities  of  greater  duration  and  collective  participation.  Clearly,  there  is  considerable  room  to 
increase  the  frequency  of  these  attributes  of  effective  professional  development  in  both  district  and 
SAHE-grantee  activities.  While  reform  professional  development  is  more  effective  than  traditional 
professional  development,  the  advantage  of  reform  activities  is  explained  primarily  by  greater 
duration. 


Lessons  for  Professional  Development 
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Second,  the  program  should  develop  additional  approaches  to  targeting  teachers  in  high- 
poverty  schools.  While  increasing  the  participation  of  teachers  in  high-poverty  schools  is  an 
important  goal  of  the  legislation,  current  approaches  to  targeting  these  teachers  are  insufficient  to 
achieve  this  goal. 

Third,  federal  indicators  might  be  improved  by  specifying  more  clearly  the  dimensions  of 
high-quality  professional  development,  focusing  on  annual  improvements  rather  than  absolute  levels, 
and  setting  higher  standards.  The  federal  indicators  are  not  as  directive  as  they  might  be  in 
specifying  the  dimensions  of  high-quality  professional  development.  Our  analyses  suggest  that  the 
federal  indicators  should  guide  local  practice  toward  greater  emphasis  upon  content  knowledge, 
active  learning,  and  coherence,  and  that  districts  should  pursue  these  attributes  of  professional 
development  through  offering  activities  of  greater  duration  and  with  more  emphasis  upon  collective 
participation.  In  addition,  several  of  the  federal  indicators  are  stated  in  terms  of  absolute  levels  to  be 
achieved  by  a specified  date  (e.g.,  “at  least  50  percent  of  teachers”).  It  would  be  better  to  have 
indicators  stated  in  terms  of  the  amount  of  improvement  needed  annually,  until  an  acceptably  high 
level  of  performance  is  reached.  Finally,  the  federal  indicators  set  some  standards  that  are  too  low. 
For  example,  in  targeting  teachers  from  high-poverty  schools,  the  standard  is  simply  to  exceed  the 
national  average.  Furthermore,  in  the  targeting  indicator,  participation  in  Eisenhower  professional 
development  is  stated  as  a dichotomous  variable  (participate  or  not),  while  clearly  teachers 
participate  in  Eisenhower- supported  professional  development  in  varying  amounts  and  types. 

Fourth,  the  program  should  pay  attention  to  building  district  capacity  to  foster  continuous 
improvement  efforts.  If  using  data  to  make  decisions  is  to  be  a serious  endeavor,  then  districts  may 
need  assistance  in  determining  the  types  of  data  that  would  be  useful  and  in  interpreting  them.  We 
believe  local  evaluation  can  be  done  in  ways  much  stronger  than  current  practice,  which  relies 
heavily  upon  participation  counts  and  teacher  satisfaction  surveys.  We  recommend  that  local 
evaluation  of  professional  development  focus  on  instruction,  by  assessing  the  degree  to  which  the 
professional  development  is  characterized  by  well-implemented  attributes  of  high-quality 
professional  development:  a focus  on  content,  active  learning,  and  coherence,  delivered  with 
sufficient  duration  and  collective  participation.  Such  evaluations  might  be  based  on  a variety  of 
sources  of  data,  including  well-designed  surveys  of  participating  teachers,  and  the  observation  of 
teachers  to  assess  the  extent  to  which  they  have  made  appropriate  changes  in  their  instruction. 
Although  the  assessment  of  student  achievement  might  be  used  as  part  of  a balanced  evaluation  of 
professional  development,  we  suggest  that  local  evaluation  efforts  should  not  focus  on  assessing  the 
effects  of  professional  development  on  student  achievement.  There  are  simply  too  many  intervening 
variables  between  professional  development  experiences  and  subsequent  gains  in  student 
achievement  to  make  such  studies  feasible  at  the  local  level.  Further,  collecting  and  analyzing  high- 
quality  data  on  gains  in  student  achievement  is  expensive  and  requires  technical  skills  that  may  not 
be  present  at  the  local  level.  Clearly,  there  needs  to  be  more  research  that  looks  at  the  relationships 
between  features  of  professional  development  and  gains  in  student  achievement,  but  this  is  better 
done  as  a part  of  well-designed  major  research  studies. 

Fifth,  more  information  is  needed  on  the  characteristics  and  conditions  that  give  some 
districts  the  capacity  to  provide  high-quality  professional  development.  We  have  speculated  that 
districts  could  provide  the  same  types  of  professional  development  as  do  SAHE  grantees.  There  are 
existence  proofs  of  districts  doing  exactly  that.  What  we  do  not  know  from  our  analyses  is  the 
percentage  of  districts  that  have  the  capacity  to  provide  such  professional  development.  Our  analyses 
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APPENDIX  A 


NATIONAL  PROFILE 


In  the  National  Profile,  we  collected  data  from  a nationally  representative  sample  of  districts 
and  SAHE  grantees  that  received  Eisenhower  funds  during  the  1997-98  school  year.  In  each 
sampled  district  and  SAHE  grantee,  we  conducted  a telephone  interview  with  the  district  Eisenhower 
coordinator  or  SAHE-granteee  project  director,  and  we  obtained  a complete  list  of  professional 
development  activities  supported  with  Eisenhower  funds  over  the  period  from  July  1 through 
December  31,  1997.  We  then  drew  a random  sample  of  these  Eisenhower-supported  activities,  and 
we  randomly  selected  two  teachers  who  attended  each  sampled  activity  and  asked  each  sampled 
teacher  to  complete  a mailed  survey. 

In  the  sections  that  follow,  we  first  discuss  the  sample  design  and  response  rates,  the 
sampling  weights,  and  the  methods  used  in  hypothesis  tests  and  standard  error  estimation  for  the 
national  sample  of  districts.  We  then  turn  to  the  sample  of  SAHE  grantees. 

Sample  Design  and  Response  Rates  for  District  Coordinator  Interviews  and 
Teacher  Activity  Survey 

Designing  the  sample.  According  to  annual  performance  reports  completed  by  the  states, 
more  than  90  percent  of  the  elementary  and  secondary  school  districts  in  the  country  participate  in 
the  Eisenhower  program.  Because  states  distribute  Eisenhower  funds  to  districts,  the  U.S. 
Department  of  Education  does  not  maintain  a list  of  all  participating  districts.  Thus,  we  based  the 
district  sampling  frame  for  the  1997-98  Teacher  Activity  Survey  on  the  Common  Core  of  Data 
(CCD),  maintained  by  the  National  Center  for  Education  Statistics.  At  the  time  we  selected  the 
sample,  the  most  recent  year  for  which  complete  CCD  data  were  available  was  1992-93.  According 
to  the  CCD,  there  were  14,645  districts  in  the  1992-1993  school  year.  Exhibit  A.l  displays  the 
universe  of  districts  by  the  number  of  teachers  employed  and  poverty  status,  and  Exhibit  A.2  displays 
the  distribution  of  public  school  teachers.  The  percentage  of  students  in  poverty  is  based  on  1990 
Census  data,  aggregated  to  the  district  level  by  the  National  Center  for  Education  Statistics.  The  cut 
points  for  the  three  poverty  status  groups  were  chosen  to  divide  the  national  population  of  teachers 
into  thirds.1 

The  data  in  the  two  tables  indicate  that  district  size  (as  measured  by  the  number  of  teachers) 
is  highly  skewed.  About  half  of  the  districts  in  the  country  have  fewer  than  60  teachers,  but  these 
districts  as  a group  account  for  less  than  10  percent  of  the  teachers.  On  the  other  hand,  just  over  5 
percent  of  the  districts  in  the  country  have  500  or  more  teachers,  but  such  districts  as  a group  account 
for  more  than  half  of  the  teachers  in  the  country. 


1 Data  on  the  percent  of  children  in  poverty  are  missing  for  1 ,873  districts.  We  imputed  missing  data  by  regression, 
using  other  sources  of  district  data  in  the  CCD,  including  data  on  free  and  reduced-price  lunch  participation,  dropout 
rates,  and  percent  of  nonwhite  enrollment. 


353 


EXHIBIT  A.1 


Number  of  School  Districts,  by  District  Size  and  Poverty  Status* 


Poverty  Status 


District  Size:  ? 
(Number  of  v 
Teachers);::  r • 

V Less  Than  10.9%  of 
Children  in  Poverty 

Between  10.9%  arid 
21 .4%  of  Children  in  v 
Poverty 

More  Than  21 .4%  of 
: Children  in  poverty 

;;v ; Total 

N 

mmm 

N 

(%) 

N 

(%) 

N 

(%) 

1-9 

588 

12.3% 

271 

6.0% 

478 

13.8% 

1337 

10.5% 

10-19 

415 

8.7% 

425 

9.4% 

417 

12.0% 

1257 

9.8% 

20-29 

325 

6.8% 

408 

9.1% 

331 

9.5% 

1064 

8.3% 

30-39 

317 

6.6% 

430 

9.5% 

265 

7.6% 

! 1012 

7.9% 

40-49 

252 

5.3% 

330 

7.3% 

198 

5.7% 

780 

6.1% 

50-59 

231 

4.8% 

276 

6.1% 

152 

4.4% 

659 

5.2% 

60-74 

303 

6.3% 

339 

7.5% 

203 

5.8% 

845 

6.6% 

75-99 

395 

8.3% 

427 

9.5% 

268 

7.7% 

1090 

8.5% 

100-149 

596 

12.5% 

502 

11.1% 

310 

8.9% 

1408 

11.0% 

150-199 

382 

8.0% 

275 

6.1% 

198 

5.7% 

855 

6.7% 

200-299 

410 

8.6% 

290 

6.4% 

221 

6.4% 

921 

7.2% 

300-499 

300 

6.3% 

240 

5.3% 

196 

5.6% 

736 

5.8% 

500-999 

187 

3.9% 

190 

4.2% 

122 

3.5% 

499 

3.9% 

1000-4999 

81 

1.7% 

96 

2.1% 

106 

3.1% 

283 

2.2% 

5000+ 

4 

0.0% 

10 

0.2% 

12 

0.4% 

26 

0.2% 

Total 

4,786 

100.1% 

4,509 

100.0% 

3,477 

100.0% 

12,772 

100.0% 

* Table  excludes  districts  that  are  missing  poverty  data.  See  footnote  1 . 


EXHIBIT  A.2 

Number  of  Teachers  Employed  by  Public  School  Districts,  by  District  Size 

and  Poverty  Status* 


Poverty  Status 


District  Size1 
(Number  of 
Teachers);:;, 

Less  Than  10.9%  of 
Children  in  Poverty 

Between  10.9%  and 
21.4%  of  Children  in 
Poverty 

More  Than  21.4%  of 
Children  in  poverty 

Total: 

\ - :y’ 

N». 

N 

N 

(%) 

N 

(%) 

N 

(%) 

1-9 

2,196 

0.3% 

1,404 

0.2% 

2,025 

0.3% 

5,625 

0.2% 

10-19 

5,763 

0.8% 

6,079 

0.8% 

5,906 

0.8% 

17,748 

0.8% 

20-29 

7,727 

1.0% 

9,586 

1.3% 

7,777 

1.0% 

25,090 

1.1% 

30-39 

10,824 

1.4% 

14,635 

1.9% 

9,097 

1.2% 

34,556 

1.5% 

40-49 

11,024 

1.5% 

14,492 

1.9% 

8,664 

1.1% 

34,180 

1.5% 

50-59 

12,469 

1.7% 

14,871 

2.0% 

8,233 

1.1% 

35,573 

1.6% 

60-74 

20,115 

2.7% 

22,538 

3.0% 

13,450 

1.8% 

56,103 

2.5% 

75-99 

34,074 

4.5% 

36,554 

4.8% 

22,937 

3.0% 

93,565 

4.1% 

100-149 

72,692 

9.7% 

60,811 

8.0% 

38,103 

5.0% 

171,606 

7.6% 

150-199 

65,846 

8.8% 

47,494 

6.3% 

34,042 

4.5% 

147,382 

6.5% 

200-299 

99,177 

13.2% 

70,505 

9.3% 

54,279 

7.2% 

223,961 

9.9% 

300-499 

115,453 

15.4% 

91,695 

12.1% 

74,274 

9.8% 

281,422 

12.4% 

500-999 

125,039 

16.7% 

128,756 

17.0% 

82,304 

10.9% 

336,099 

14.8% 

1000-4999 

141,556 

18.9% 

171,126 

22.6% 

218,500 

28.8% 

531,182 

23.4% 

5000+ 

25,301 

3.4% 

67,609 

8.9% 

178,376 

23.5% 

271,286 

12.0% 

Total 

749,256 

100.0% 

758,155 

100.0% 

757,967 

100.0% 

2,265,378 

100.0% 

* Table  excludes  districts  that  are  missing  poverty  data.  See  footnote  1. 
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Because  district  size  is  highly  skewed,  a simple  random  sample  would  contain  many  small 
districts  representing  very  few  teachers.  Thus,  we  drew  the  sample  of  districts  with  probability 
proportional  to  district  size,  separately  within  each  of  the  three  poverty  strata,  using  the  number  of 
teachers  employed  as  the  measure  of  size.  This  approach  allowed  us  to  obtain  efficient  estimates  of 
Eisenhower  program  characteristics,  weighting  districts  in  proportion  to  the  number  of  teachers 
employed.  Within  each  stratum,  we  selected  with  certainty  all  districts  with  5,000  or  more 
teachers — four  low-poverty  districts,  10  medium-poverty  districts,  and  12  high-poverty  districts. 

To  obtain  sufficiently  precise  estimates  of  program  characteristics,  we  planned  to  conduct 
interviews  with  400  district  coordinators.  Based  on  pilot  interviews  in  a small  sample  of  districts,  we 
learned  that,  in  some  large  districts,  Eisenhower  funds  are  divided  among  subdistricts.  When 
districts  divide  funds  among  subdistricts,  we  concluded  that  it  might  be  necessary  to  conduct  separate 
interviews  and  obtain  separate  activity  lists  within  each  subdistrict.  In  such  cases,  each  subdistrict 
would  operate,  in  effect,  as  a separate  district.  To  estimate  the  number  of  subdistrict  interviews  that 
might  be  required,  we  assumed  that  subdistrict  interviews  would  occur  in  only  very  large  districts 
(i.e.,  the  26  districts  with  more  than  5,000  teachers),  and  we  assumed  that  one  subdistrict  interview 
would  be  required  per  5,000  teachers.  These  calculations  led  us  to  estimate  that  the  26  certainty 
districts  in  our  sample  might  generate  53  interviews  altogether — 27  more  than  would  be  required 
without  subdistricts. 

Since  we  desired  an  overall  sample  size  of  400  interviews,  we  set  a total  sample  size  of  373 
districts,  to  accommodate  the  anticipated  27  additional  subdistrict  interviews.  Because  we  planned  to 
select  26  districts  with  certainty,  this  left  347  districts  to  be  drawn  with  probability  proportional  to 
size.  We  allocated  these  347  districts  to  the  three  strata  in  proportion  to  each  stratum’s  total  number 
of  teachers  in  districts  with  fewer  than  5,000  teachers.  This  procedure  yielded  a sample  size  of  140 
low  poverty  districts,  129  medium-poverty  districts,  and  104  high-poverty  districts.2 

Screening  districts  and  scheduling  interviews.  After  drawing  an  initial  sample  of  districts, 
we  contacted  the  states  in  which  the  sample  districts  were  located  to  check  whether  the  sampled 
districts  participated  in  the  Eisenhower  program  and  to  obtain  the  name  and  telephone  number  of  the 
district  Eisenhower  coordinator.  About  seven  percent  of  districts  did  not  participate.  We  replaced 
non-participating  districts  that  fell  into  the  sample  with  randomly  drawn  districts  of  similar  size  and 
poverty  status. 

Finally,  we  contacted  the  districts  drawn  into  the  sample  to  invite  them  to  participate  in  the 
evaluation.  If  a district  refused,  we  replaced  the  district  with  a randomly  drawn  district  of  similar 
size  and  poverty  status.3  In  all,  we  contacted  409  districts  that  received  Eisenhower  funds  and  were 
thus  eligible  to  participate  in  the  study. 


2 The  variation  in  sample  size  across  the  three  strata  is  a consequence  of  differences  across  the  strata  in  the  size  of 
the  certainty  districts.  Although  all  three  strata  have  about  750,000  teachers,  178,376  of  the  teachers  in  the  high- 
poverty  stratum  are  in  certainty  districts,  while  only  67,609  of  the  teachers  in  the  medium-poverty  stratum  are  in 
certainty  districts.  Since  the  non-certainty  districts  were  drawn  with  probability  proportional  to  size,  this  implies 
that  more  non-certainty  districts  were  required  in  the  medium-poverty  stratum  than  in  the  high-poverty  stratum. 

3 Only  one  of  the  26  certainty  districts  refused,  and  it  was  replaced  with  the  largest  available  non-certainty  district 
that  had  not  already  been  drawn  into  the  sample. 
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When  we  contacted  each  sampled  district,  we  asked  the  district  if  it  participated  as  an 
individual  district  or  as  a member  of  a consortium.4 * 6  For  districts  that  were  members  of  consortia,  we 
treated  the  consortium  rather  than  the  district  as  the  unit  of  analysis.  For  each  consortium  included  in 
our  sample,  we  obtained  a list  of  the  districts  participating  in  the  consortium,  along  with  the  number 
of  teachers  each  district  employed.  We  then  used  these  data  to  determine  the  probability  with  which 
each  consortium  was  drawn.5-6  Most  of  the  districts  with  enrollments  below  1,500  that  were  drawn 
into  the  sample  were  members  of  consortia. 

In  addition,  during  our  initial  contact  with  each  district,  we  confirmed  the  name  of  the  district 
Eisenhower  coordinator  and  conducted  a screening  interview  to  determine  whether  the  Eisenhower 
program  in  the  district  was  administered  through  subdistricts,  and,  if  so,  whether  separate  subdistrict 
interviews  would  be  required.  The  screening  interviews  indicated  a number  of  districts  administered 
the  program  through  subdistricts,  but  separate  subdistrict  interviews  would  be  needed  only  in  one 
large  certainty  district.  Thus,  in  this  large  district,  we  sampled  1 1 subdistricts — the  expected  number 
that  would  have  been  drawn  into  the  sample  had  each  subdistrict  been  a separate  district  in  the 
original  sampling  frame.  As  a consequence,  the  total  number  of  districts  and  subdistricts  we  asked  to 
participate  in  the  study  was  419 — 408  districts  and  1 1 subdistricts. 

During  the  screening  interview,  we  asked  whether  the  Eisenhower  program  was  administered 
by  a single  coordinator  in  the  district,  or  whether  funds  were  allocated  to  be  administered  separately 
by  distinct  organizational  units,  for  example,  the  offices  of  a mathematics  currciulum  coordinator  and 
science  curriculum  coordinator.  In  15  districts  in  which  funds  were  allocated  to  separate 
organizational  units,  we  learned  that  we  would  need  to  conduct  separate  interviews  with  each  unit 
and  obtain  separate  activity  lists.  Of  these  15  districts,  12  involved  two  administrators,  one  involved 
three,  and  two  involved  four,  producing  35  potential  interviews  in  all.7  Thus,  the  419  districts  and 
subdistricts  we  contacted  generated  a total  of  439  potential  interviews,  including  the  35  interviews  in 
the  15  districts  that  administered  their  programs  in  separate  organizational  units. 

Response  rates  for  district  interviews.  We  attempted  to  schedule  and  complete  an  hour- 
long  computer-assisted  telephone  interview  (CATI)  with  each  of  the  439  administrators  identified 
through  the  screening  process.  Of  these  439  potential  interviews,  we  completed  386,  for  an  overall 
interview  response  rate  of  88  percent. 


4 Districts  that  receive  less  than  $10,000  are  required  to  collaborate  with  other  districts,  in  consortia,  in  order  to 
receive  program  funds,  unless  this  requirement  is  explicitly  waived.  Most  districts  with  enrollments  below  1,500 
that  participate  in  the  Eisenhower  program  do  so  as  members  of  consortia.  Each  consortium  operates  as  an 
administrative  unit  with  a single  Eisenhower  budget,  program,  and  coordinator. 

Since  each  consortium  drawn  into  the  sample  would  have  been  included  had  any  of  its  member  districts  been 
drawn,  the  probability  of  drawing  each  consortium  is  equal  to  the  sum  of  the  individual  sampling  probabilities  of  the 
consorium’s  member  districts. 

6 We  estimated  the  percent  of  school-age  children  in  poverty  for  each  consortium  by  computing  the  weighted 
average  of  the  percent  of  school-age  children  in  poverty  in  the  member  districts,  weighting  each  district  by  the 
number  of  teachers  employed. 

In  about  ten  other  districts  in  which  the  administration  of  the  program  was  split  among  separate  units,  the 
administrators  involved  decided  to  participate  in  the  interview  as  a group. 
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Of  the  completed  interviews,  32  were  in  districts  in  which  the  program  was  administered  by 
separate  organizational  units.8  In  our  analyses,  we  collapsed  the  multiple  interviews  conducted  in 
each  of  these  districts  into  a single  district-level  response  for  each  item 9 

Altogether,  we  completed  interviews  in  369  districts  and  consortia,  of  the  419  districts  and 
subdistricts  in  which  we  attempted  interviews.  This  produces  a district-level  response  rate  of  88 
percent.  This  response  rate,  based  on  the  number  of  responding  districts  (369  of  419),  is  nearly 
identical  to  the  response  rate  based  on  the  number  of  interviews  completed  (386  of 439),  which  we 
discussed  above.  Of  the  369  completed  interviews,  six  were  unreadable  due  to  a malfunction  of  the 
Computer-assisted  Telephone  interviews  (CATI)  software,  leaving  363  in  our  basic  analytic 
sample.10 

Exhibit  A.3,  reports  the  realized  sample  size  by  district  size  (number  of  teachers)  and  poverty 
status.11,12 

We  examined  the  response  rates  for  the  district  telephone  interviews  to  determine  whether 
response  rates  differ  by  district  size  or  poverty.  We  observe  a moderate-size  effect:  large  districts 
and  consortia  were  somewhat  more  likely  to  complete  a coordinator  interview  than  small  districts 
(p<.05;  see  Exhibit  A.4).  We  did  not  observe  an  effect  of  district  poverty  on  response  rates. 


8 Interviews  with  at  least  one  administrative  unit  were  completed  in  each  of  the  15  districts  in  which  the  program  is 
administered  by  separate  units. 

9 For  items  asking  for  a total  (for  example,  the  total  number  of  activities  supported  with  Eisenhower  funds),  we 
summed  across  the  separate  interviews  to  obtain  a district-level  response.  For  items  asking  for  a typical  or  average 
response,  we  averaged  the  responses  across  the  separate  interviews.  For  items  asking  whether  a particular  type  of 
activity  or  event  occurred  (for  example,  study  groups  or  mentoring),  we  generated  a district-level  response  by 
examining  whether  the  event  in  question  occurred  in  any  of  the  separate  interviews. 

10  The  six  lost  interviews  can  be  viewed  as  a random  draw  from  the  completed  set  of  369.  If  the  six  lost  interviews 
are  considered  as  reducing  the  size  of  the  originally  drawn  sample,  our  response  rate  can  be  computed  as  363 
respondents  from  among  413  intended  interviews,  which  produces  a response  rate  of  88  percent.  If  they  are 
considered  as  non-respondents,  our  response  rate  drops  to  87  percent. 

11  For  each  sampled  district  that  was  a member  of  a consortium,  the  district  size  shown  in  the  table  is  the  sum  of  the 
number  of  teachers  in  all  districts  participating  in  the  consortium.  Two  consortia  have  a size  of  more  than  5000 
teachers. 

12  Of  the  26  certainty  districts  in  the  sample,  one  did  not  respond.  Also,  as  discussed  above,  for  one  large  certainty 
district,  we  subsampled  1 1 community  districts  (subdistricts).  Each  responding  community  district  in  New  York 
appears  as  separate  “district”  in  Exhibit  A.3.  Thus,  the  total  number  of  districts  with  size  above  5000  shown  in  the 
Exhibit  is  26  - 24  of  the  certainty  districts  in  the  sample,  plus  two  consortia  (see  note  1 1). 
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EXHIBIT  A.3 


Number  of  Districts  That  Responded  to  the  District  Coordinator  Telephone  Survey,  by 

Size  and  Poverty  Status 


Poverty  Status 

District  Size:  * ' 
(Numberofv ; • 
Teachers) 

Less  Than10:9%  of 
Children  in  Poverty 

Between  1 0.9%  arid  =# 
21 .4%  of  Children  in  > 
Poverty  • 

\i:  More  Than  21 .4%  of 
Children  in  Poverty 

v;;:;:  Total 

N 

(%) 

N 

(%) 

N 

(%) 

N 

(%) 

1-9 

0 

0.0% 

0 

0.0% 

1 

1.0% 

i 

0.3% 

10-19 

1 

0.7% 

2 

1.5% 

0 

0.0% 

3 

0.8% 

20-29 

1 

0.7% 

0 

0.0% 

0 

0.0% 

1 

0.3% 

30-39 

1 

0.7% 

1 

0.8% 

1 

1.0% 

3 

0.8% 

40-49 

1 

0.7% 

0 

0.0% 

0 

0.0% 

1 

0.3% 

50-59 

2 

1.5% 

2 

1.5% 

1 

1.0% 

5 

1.4% 

60-74 

1 

0.7% 

1 

0.8% 

2 

2.1% 

4 

1.1% 

75-99 

3 

2.2% 

6 

4.6% 

3 

3.1% 

12 

3.3% 

100-149 

11 

8.1% 

8 

6.2% 

10 

10.3% 

29 

8.0% 

150-199 

12 

8.8% 

8 

6.2% 

3 

3.1% 

23 

6.3% 

200-299 

16 

11.8% 

11 

8.5% 

4 

4.1% 

31 

8.5% 

300-499 

17 

12.5% 

13 

10.0% 

12 

12.4% 

42 

11.6% 

500-999 

23 

16.9% 

23 

17.7% 

6 

6.2% 

52 

14.3% 

1000-4999 

43 

31.6% 

43 

33.1% 

44 

45.4% 

130 

35.8% 

5000+ 

4 

2.9% 

12 

9.2% 

10 

10.3% 

26 

7.2% 

Total 

136 

100.0% 

130 

100.0% 

97 

100.0% 

363 

100.0% 

EXHIBIT  A.4 

Response  Rates  for  District  Interview,  by  District  Size 


District  size 

Number  of  completed 
telephone  interviews* 

Percent  of  intended 
sample* 

Small  (1-249  teachers) 

98 

82% 

Medium  (250-499  teachers) 

130 

87% 

Large  (500  or  more  teachers) 

98 

94% 

Consortium 

37 

95% 

Total 

* kl l t 1 a _J  ■ X : _ _ 

363 

88% 

be  included  in  final  data  set. 


Sampling  activities  and  teachers.  Within  each  district  in  which  we  completed  an  interview, 
we  asked  the  Eisenhower  coordinator  to  provide  a complete  list  of  all  Eisenhower-assisted  activities 
conducted  in  the  district  over  the  period  from  July  1 through  December  31,  1997.  Ten  of  the  369 
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districts  with  completed  interviews  did  not  conduct  any  Eisenhower-assisted  activities  during  this 
period,  leaving  359  from  which  we  potentially  could  obtain  activity  lists.13 

For  most  districts,  we  drew  two  activities  at  random  from  the  complete  list  of  mathematics 
and  science  activities  provided,  with  probability  proportional  to  the  number  of  participants  in  each 
activity.14  For  districts  with  more  than  7,500  teachers,  we  drew  one  activity  for  each  2,500  teachers 
teaching  in  the  district.15  Thus,  for  example,  we  drew  three  activities  in  districts  with  between  7,500 
and  9,999  teachers,  and  four  activities  in  districts  with  between  10,000  and  12,249  teachers. 

Once  the  sample  of  activities  was  drawn  for  each  district,  we  asked  the  district  Eisenhower 
coordinator  to  provide  a list  of  all  teachers  participating  in  each  of  the  sampled  activities.  We  then 
drew  a simple  random  sample  of  two  teachers  for  each  of  the  selected  activities.1617  Of  the  359 
districts  from  which  we  attempted  to  obtain  activity  lists  and  teacher  names,  we  obtained  complete 
activity  lists  and  teacher  names  from  312,  a response  rate  of  87  percent.18  We  examined  variation  in 
response  rates  for  activity  lists  and  teacher  names  by  district  size  and  poverty.  We  observed  some 
differences  in  the  response  rates  for  activity  lists  by  district  size,  but  there  is  no  clear  pattern  (see 
Exhibit  A. 5).  We  did  not  observe  any  differences  in  response  by  poverty. 

EXHIBIT  A.5 

Response  Rates  for  Activity  Lists  and  Teacher  Names,  by  District  Size 


District  size 

Number  of  districts 
providing  activity  lists 
and  teacher  names* 

Percent  of  intended 
sample* 

Small  (1-249  teachers) 

77 

84% 

Medium  (250-499  teachers) 

121 

93% 

Large  (500  or  more  teachers) 

80 

80% 

Consortium 

34 

92% 

Total 

312 

87% 

* Intended  sample  includes  359  districts  that  completed  a telephone  interview  and  conducted  activities  over  the 
period  from  July  1 through  December  31 , 1997. 


13  The  ten  districts  that  received  Eisenhower  funds  for  the  1997-98  school  year  but  did  not  conduct  activities  over 
the  period  from  July  through  December  1997  expected  to  conduct  activities  later  in  the  school  year. 

14  Because  our  questionnaire  items  asked  explicitly  about  the  content  of  the  activity,  it  was  necessary  to  restrict  our 
focus  to  mathematics  and  science  activities. 

15  We  selected  more  than  two  activities  in  the  largest  certainty  districts  to  maintain  roughly  equal  teacher  selection 
probabilties  across  districts.  See  the  discussion  of  sampling  weights,  below. 

6 To  facilitate  the  process  of  obtaining  teacher  names  for  each  sampled  activity,  we  gave  district  coordinators  the 
option  of  sending  us  a complete  list  of  teacher  names  for  the  sampled  activity,  or  instead  giving  us  the  names  of  two 
teachers  appearing  in  specific  randomly  determined  positions  on  the  list  of  teachers.  (For  example,  if  an  activity 
included  15  teachers,  and  our  random  selection  process  determined  that  the  5th  and  12th  teachers  should  be  drawn, 
we  asked  the  coordinator  to  count  down  the  list  and  give  us  the  names  of  the  5lh  and  12th  teachers.) 

17  In  drawing  the  sample  of  individuals  to  receive  the  teacher  survey,  we  excluded  participants  other  than  teachers 
(for  example,  administrators  or  paraprofessionals). 

18  This  response  rate  is  based  on  districts  that  completed  interviews  and  conducted  activities  over  the  period  from 
July  1,  1997  through  December  31,  1997.  For  the  full  intended  sample  of  409  districts,  less  10  that  did  not  offer 
activities  during  the  relevant  period,  the  response  rate  for  activity  lists  and  teacher  names  is  78  percent. 
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Response  rates  for  teachers.  Altogether,  for  the  312  districts  for  which  we  obtained  activity 
lists  and  teacher  names,  we  sampled  1,255  teachers.19,20  We  obtained  the  school  address  for  each  of 
these  1 ,255  teachers  from  the  Eisenhower  coordinators,  and  we  mailed  either  a mathematics  or 
science  form  of  the  teacher  activity  survey,  depending  on  the  type  of  activity  attended. 

Some  teachers  who  received  a survey  did  not  remember  attending  the  activities  that  drew 
them  into  the  sample.  For  example,  some  teachers  indicated  that  they  had  been  scheduled  to  attend, 
but  were  unable  to  participate.  In  addition,  some  addresses  we  obtained  from  the  district  Eisenhower 
coordinators  were  in  error,  because  the  participating  teachers  had  changed  schools  or  left  teaching 
since  attending  the  sampled  activities.  Finally,  a few  of  the  sampled  teachers  were  seriously  ill  or  out 
of  the  country  and  could  not  be  contacted.  Based  on  a careful  effort  to  locate  and  contact  each 
sampled  teacher  by  telephone,  we  concluded  that  142  of  the  1,255  sampled  teachers  (or  1 1 percent) 
were  out  of  scope,  because  they  had  not  attended  the  sampled  activities,  or  they  could  not  be  located, 
because  the  address  provided  by  the  Eisenhower  coordinator  was  in  error  and  we  were  unable  to 
obtain  a current  address.21  Of  the  remaining  1,113  potential  respondents,  we  received  completed 
surveys  from  783,  or  just  over  70  percent. 

We  checked  whether  teacher  response  rates  differed  by  district  size  and  poverty,  as  well  as 
for  teachers  who  received  mathematics  and  science  surveys,  and  for  teachers  in  high-  and  low- 
poverty  schools.  We  did  not  find  significant  differences  for  any  of  these  factors. 

Sampling  Probabilities  and  Weights  for  the  District  Coordinator  Interview 

The  final  weights  used  in  our  analyses  of  the  district  coordinator  interview  data  reflect  two 
components:  a sampling  weight,  designed  to  incorporate  the  fact  that  districts  in  our  sample  were 
selected  with  unequal  probabilities,  and  a weight  equal  to  the  number  of  teachers  in  each  district, 
included  so  that  our  reported  results  represent  the  role  of  the  Eisenhower  program  in  proportion  to 
the  number  of  teachers  in  the  districts  in  which  the  program  operates. 

As  discussed  above,  26  districts  (those  with  5,000  or  more  teachers)  were  included  in  the 
national  profile  with  certainty;  the  remaining  sample  of  districts  was  drawn  with  probability 
proportional  to  size.  The  final  weights  differ  for  the  non-certainty  and  certainty  districts,  and  thus  we 
will  consider  each  in  turn. 

Non-certainty  districts.  In  a sample  drawn  with  probability  proportional  to  size,  the 
probability  of  selection  for  district  k (n^)  can  be  expressed: 


19  This  number  of  teachers  sampled  is  larger  than  four  times  the  number  of  districts  providing  activity  lists  and 
teacher  names,  because  we  sampled  more  than  two  activities  per  district  in  large  districts. 

Although  we  intended  to  draw  two  activities  per  district  and  two  teachers  per  activity  for  all  districts  with  fewer 
than  5,000  teachers,  in  some  cases,  our  final  sample  of  teachers  per  district  was  smaller  than  four.  In  a few  districts, 
only  one  activity  in  mathematics  or  science  was  offered  over  the  relevant  period,  and  thus,  in  these  districts,  we 
sampled  only  two  teachers.  Also,  in  a few  districts,  only  one  teacher  participated  in  a sampled  activity. 

If,  during  the  period  in  which  we  conducted  the  telephone  interviews,  we  learned  that  a sampled  teacher  was  out 
of  scope,  we  replaced  the  teacher  with  another  randomly  drawn  teacher  from  the  same  activity.  If  we  did  not  learn 
that  a teacher  was  out  of  scope  until  we  had  completed  the  telephone  interviews,  we  did  not  replace  the  teacher. 
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where  n is  the  number  of  districts  in  the  national  sample,  Tk  is  the  number  of  teachers  in  district  k, 
and  T is  the  total  number  of  teachers  in  all  districts  in  the  population. " 


The  sampling  weight  for  district  k (wk)  is  given  by  the  inverse  of  the  sampling  probability: 


w. 


_1 T_ 

< " nTk 


The  final  district  weight  involves  the  product  of  the  sampling  weight  wk  and  the  number  of  teachers 
Tk: 


/* 


= wJk  = 


T 

n 


Thus,  for  non-certainty  districts,  the  combination  of  sampling  with  probability  proportional  to  size 
and  weighting  by  the  number  of  teachers  produces  equal  weights  for  all  districts.23 

Certainty  districts.  For  each  district  drawn  with  certainty,  the  sampling  weight  (i.e.,  the 
inverse  of  the  selection  probability)  is  one.  Thus,  the  final  weight  for  each  certainty  district  is  simply 
the  number  of  teachers  in  the  district,  7*.24 

Sampling  Probabilities  and  Weights  for  the  District  Teacher  Activity  Survey 

We  generally  sampled  two  Eisenhower-supported  activities  in  each  sampled  district  and  two 
teachers  per  sampled  activity,  as  described  above.  The  sampling  plan  was  formulated  to  give  all 
teachers  in  Eisenhower-supported  activities  approximately  equal  probabilities  of  selection.  The 
selection  probability  for  teacher  i in  activity  j in  district  A:  is  a function  of  three  terms:  the  probability 
of  selection  of  district  k (discussed  above),  the  probability  of  selection  of  activity  j,  given  that  district 
k has  been  selected,  and  the  probability  of  selection  of  teacher  i within  activity  j. 

We  consider  non-certainty  districts  first,  and  then  turn  to  certainty  districts. 

Non-certainty  districts.  The  probability  of  selection  for  activity  j,  given  that  district  k has 
been  selected,  is: 

—U  _ 

jk\k  'Y'E 
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where  ty  is  the  total  number  of  teachers  who  participated  in  activity  j in  district  k and  7*£  is  the  total 
number  of  teachers  who  participated  in  all  EPDP-supported  activities  in  district  k. 

The  probability  of  selection  of  teacher  /,  given  that  activity  j in  district  k has  been  selected,  is: 


22  To  compute  the  probabilities,  we  set  n=347  (the  number  of  non-certainty  districts)  and  T=  1,994,092,  the  number 
of  teachers  in  districts  with  fewer  than  5,000  teachers. 

23  With  n=347  and  T=  1,994, 0992,  the  constant  weight  = 5575. 

24  The  weights  for  certainty  districts  vary  from  a minimum  of  about  5,000,  for  the  smallest  certainty  district,  to  approximately 
25,000,  for  Los  Angeles. 
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where  tjk  is  the  total  number  of  teachers  who  participated  in  activity  j in  district  k. 
Thus,  the  selection  probability  for  teacher  i in  activity  j in  district  k is: 
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Hence,  the  sampling  weight  w,y*  for  teacher  i in  activity  j in  district  k can  be  expressed: 


wijk  = 


V 

A 


rp  E ^ 


As  can  be  seen,  the  sampling  weight  is  independent  of  the  teacher  or  the  particular  activity  in  which  a 
teacher  participates  (that  is,  the  terms  and  do  not  appear  in  the  expression  for  the  sampling 


weight).  But  the  probability  of  selection  does  vary  across  districts,  in  proportion  to  the  term 


f rrE  ^ 

Ik 


The  numerator  of  the  term  is  the  number  of  teacher  participations  in  Eisenhower-assisted  activities  in 
district  k,  and  the  denominator  is  the  total  number  of  teachers  in  the  district.  In  other  words,  the  term 


^ rp  E ^ 

*k 


is  the  teacher  participation  rate  in  Eisenhower-activities  in  the  district.  The  term  reflects  the 


fact  that  sampled  teachers  are  given  more  weight  if  they  are  in  districts  with  high  participation  rates 
than  in  districts  with  low  rates. 


To  calculate  the  sampling  weights  for  the  teachers  in  our  sample,  we  required  data  on  the 
number  of  Eisenhower  participations  in  each  district,  TkE.  For  each  district  in  our  sample,  we  used 
the  average  of  two  estimates:  the  total  number  of  participations  reported  in  the  activity  list  we 
obtained  from  the  district,  and  the  number  of  participations  reported  in  the  telephone  interview.25  A 
few  districts  (about  five  percent)  had  seemingly  extreme  values  for  one  or  the  other  of  these 
participation  rates.  We  viewed  a participation  rate  of  2.5  percent  of  the  teachers  in  a district  as  a 
plausible  lower  bound  on  the  participation  rate,  and  a participation  rate  of  200  percent  as  a plausible 
upper  bound.26  If  one  of  the  two  estimated  rates  for  a district  was  outside  these  bounds  and  the  other 
was  inside,  we  chose  the  value  inside.  If  both  values  were  outside,  we  Winsorized  the  estimate,  i.e., 
we  chose  the  lower  bound  if  both  values  were  low,  and  the  upper  bound  if  both  values  were  high. 


25  We  assumed  that  each  of  these  two  sources  was  somewhat  unreliable,  and  thus  the  average  of  the  two  would  be 
more  reliable  than  either  alone. 

26  A participation  rate  of  200  percent  might  imply  that  each  teacher  in  a district  participated  in  two  Eisenhower- 
assisted  activities. 
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Certainty  districts.  For  certainty  districts,  the  number  of  activities  we  sampled  was  a 
function  of  district  size.  The  probability  of  selection  for  activity  j,  given  that  district  k has  been 
selected,  can  be  written: 
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where  mk  is  the  total  number  of  activities  selected  in  district  k,  t]k  is  the  total  number  of  teachers  who 
participated  in  activity  j in  district  k,  and  7 \E  is  the  total  number  of  teachers  who  participated  in  all 
Eisenhower-supported  activities  in  district  k.  We  chose  nik,  the  number  of  activities  sampled  in 
district  k,  based  on  the  number  of  teachers  in  the  district,  Tk.  In  particular,  we  selected  2 teachers  in 
districts  with  between  5,000  and  7,500  teachers,  3 in  districts  with  between  7,500  and  10,000 
teachers,  and  so  forth.  Thus,  in  approximate  terms: 


m 
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_ 2 *Tk 
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Hence,  the  probability  of  selection  for  activity  j,  given  that  district  k has  been  selected,  can  be 
expressed: 
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15000, 

The  probability  of  selection  of  teacher  i,  given  that  activity  j in  district  k has  been  selected, 
can  be  written: 


Kijk\jk  t 

Jk 


where  tjk  is  the  total  number  of  teachers  who  participated  in  activity  j in  certainty  district  k. 
Thus,  the  selection  probability  for  teacher  i in  activity  j in  district  k is: 
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Thus,  the  final  teacher  weight  for  each  certainty  district  is: 
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Like  the  final  teacher  weight  for  non-certainty  districts,  the  weight  for  certainty  districts  depends  on 
the  participation  rate,  which  we  computed  using  the  procedures  described  above  for  non-certainty 
districts. 

Methods  for  District  Hypothesis  Tests  and  Standard  Error  Estimation 

In  the  sections  below,  we  discuss  our  approach  to  estimating  standard  errors  for  the  district 
telephone  interview  and  teacher  activity  results. 

District  coordinator  interviews.  In  the  analyses  of  the  district  coordinator  interview  data 
reported  in  Chapters  4 and  5,  the  significance  tests  are  computed  according  to  the  assumptions  of 
conventional  simple  random  sampling.  These  tests  do  not  take  into  account  the  fact  that  there  is 
modest  variability  in  the  weights  (i.e.,  the  largest  certainty  districts  have  larger  weights  than  the  other 
districts  in  the  sample).  Taking  the  variability  in  the  weights  into  account  would  produce  standard 
errors  slightly  larger  than  those  upon  which  the  reported  significance  tests  are  based27  We  have 
rerun  the  analyses  incorporating  the  variability  in  weights,  and  the  significance  test  results  are  nearly 
identical  to  those  reported  in  Chapters  3 and  4. 

Teacher  surveys.  Our  analysis  of  the  characteristics  of  Eisenhower-supported  activities 
discussed  in  Chapter  3 rely  on  teacher-reported  data.  The  significance  tests  currently  presented  in 
Chapter  3 are  computed  according  to  the  assumptions  of  simple  random  sampling.  More  precise 
tests  can  be  conducted  that  take  into  account  three  factors  not  currently  considered  in  the  reported 
results:  the  fact  that  the  sample  is  stratified  by  poverty,  the  fact  that  the  final  weights  vary  across 
teachers  in  different  districts  (see  the  discussion  above),  and  the  fact  that  the  reports  of  teachers  who 
attended  the  same  activities  or  attended  activities  in  the  same  districts  are  not  independent.28  The 
first  of  these  factors  would  produce  somewhat  smaller  standard  errors  than  those  upon  which  the 
reported  significance  tests  are  based,  while  the  second  and  third  would  produce  standard  errors 
somewhat  larger.  We  have  rerun  the  analyses  to  incorporate  these  factors,  and  the  significance  test 
results  are  very  similar  to  those  reported  in  Chapter  3. 

Sample  Design  and  Response  Rates  for  IHE/NPO  Project  Director  Interviews 
and  Teacher  Activity  Survey 

Designing  the  sample.  In  most  respects,  the  approach  we  used  in  sampling  institutions  of 
higher  education/not-for-profit  programs  paralleled  the  process  for  districts.  Thus,  in  our  discussion 
below,  we  focus  mainly  on  the  differences. 


27  All  analyses  reported  in  Chapters  4 and  5 take  into  account  the  fact  that  the  sample  is  stratified  by  poverty.  All 
reported  analyses  of  variance  explicitly  include  poverty  as  a factor,  and  the  regression  analyses  include  the  percent 
of  school-age  children  in  poverty  as  a control. 

While  the  analyses  reported  in  Chapter  3 include  school  poverty  (i.e.,  percent  of  students  eligible  for  free  lunch) 
as  a control,  they  do  not  include  district  poverty  (district  percent  of  school  age  children  in  poverty). 


The  primary  difference  in  our  sampling  plans  for  SAHE  grantees  and  districts  is  in  our 
measure  of  size.  For  districts,  as  discussed  above,  we  sampled  with  probability  proportional  to  the 
number  of  teachers  in  each  district,  with  the  expectation  that  the  number  of  teachers  would  be 
proportional  to  the  number  of  Eisenhower  participants.  For  SAHE  grantees,  we  could  not  use  the 
number  of  teachers  as  a basis  for  selection,  since  SAHE  grantees  are  not  staffed  with  teachers,  as  are 
school  districts.  Furthermore,  we  could  not  sample  based  directly  on  the  number  of  participants 
enrolled  in  Eisenhower-assisted  SAHE-grantee  activities,  because  information  on  the  number  of 
participants  is  not  available  on  the  full  population  of  SAHE  grantees  receiving  Eisenhower  funds. 
Hence,  we  sampled  SAHE  grantees  based  on  the  size  of  the  Eisenhower  award  each  received,  in 
dollars  per  year.  We  reasoned  that,  while  the  award  might  not  be  perfectly  correlated  with  the 
number  of  participants,  it  would  serve  as  a reasonable  basis  for  sampling. 

For  purposes  of  the  sampling  plan,  we  defined  the  population  of  SAHE  grantees  as  all 
institutions  with  an  Eisenhower  grant  covering  at  least  part  of  the  1997-98  school  year,  and  offering 
at  least  one  Eisenhower-assisted  activity  during  the  period  from  July  1 through  December  31,  1997 ,29 
To  construct  a sampling  frame  of  the  SAHE  grantees  that  received  funds  for  the  1997-98  school  year, 
we  contacted  the  50  state  agency  for  higher  education  (SAHE)  Eisenhower  coordinators  and  asked 
them  to  provide  a list  of  all  SAHE  grantees  that  had  been  awarded  grants,  along  with  information  on 
the  size  of  each  grant  and  the  grant  period,  in  months.  All  states  provided  appropriate  information, 
which  is  summarized  in  Exhibit  A.6,  below. 


EXHIBIT  A.6 

Number  of  IHE  Eisenhower  Programs  and  Number  in  National  Profile  Sample,  by  Size  of 

Grant  Received,  in  Dollars  per  Year 


Size  of  Grant 
(dollars  per  year) 

Universe  of 
Institutions 

Percent  of 
Institutions 

Percent  of 
Funds 

Number  of : v; 
Institutions,. 4 
in  Sample 

Percent  of:. 
Institutions  i 
in  Sample 

Less  than  $10,000 

44 

5.0 

0.8 

1 

1.1 

$10,000 -$24,999 

220 

30.3 

12.3 

13 

14.1 

$25,000  - $49,999 

257 

35.5 

28.3 

25 

27.2 

$50,000  - $99,999 

164 

22.6 

35.0 

36 

39.1 

$100,000  or  more 

48 

6.6 

23.6 

17 

18.5 

Total 

725 

100.0 

ioo.o 

92 

100.0  : : 

Screening  SAHE  grantees  and  scheduling  interviews.  We  sought  a completed  sample  size 
of  100  SAHE  grantees.  Once  the  initial  sample  was  drawn,  we  contacted  each  sampled  institution  to 
request  participation  in  the  evaluation.  As  part  of  the  screening  process,  we  determined  whether  each 
sampled  institution  had  offered  Eisenhower-assisted  activities  during  the  period  from  July  1 through 
December  3 1 , 1999.  Institutions  that  did  not  or  that  declined  to  participate  were  replaced  with 
randomly  drawn  institutions  of  similar  grant  size. 

Response  rates  for  SAHE  grantees.  We  contacted  120  institutions;  of  these,  12  did  not 
conduct  activities  in  the  relevant  period  and  two  did  not  receive  Eisenhower  funds  over  the  relevant 
period,  although  they  were  included  on  the  list  of  funded  projects  provided  by  their  state.  Thus,  these 


29 IHE/NPO  grants  vary  considerably  in  the  period  of  time  covered.  Furthermore,  the  formal  grant  period  may 
extend  over  a period  substantially  longer  than  the  period  in  which  an  IHE/NPO  offers  activities.  Hence,  we 
restricted  the  population  to  SAHE  grantees  that  offered  activities  from  July  through  December  1997. 
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14  institutions  did  not  meet  the  conditions  to  be  included  in  our  sample,  and  we  considered  them  out 
of  scope.  We  replaced  each  out-of-scope  SAHE  Grantee  with  a randomly  drawn  SAHE  granteewith 
similar  annual  dollar  grant  amount.30  Of  the  remaining  106  institutions,  92  completed  interviews, 
producing  an  overall  response  rate  of  87  percent. 

Sampling  activities  and  teachers.  As  we  did  in  each  sampled  district,  we  asked  each 
sampled  SAHE  granteeto  provide  a complete  list  of  Eisenhower-assisted  activities  that  the  institution 
had  offered  over  the  period  from  July  1 through  December  31,  1997,  and,  from  these  lists,  we 
selected  activities  to  be  included  in  our  sample.  We  had  intended  to  follow  the  plan  we  used  for 
districts,  and  sample  two  activities  per  SAHE  grantee.  But  as  we  began  to  receive  activity  lists  from 
the  SAHE  grantees,  we  learned  that  many  SAHE  grantees  in  our  sample  offered  just  one  activity  over 
the  relevant  period,  and  most  that  offered  more  than  one  activity  offered  a relatively  small  number. 
Thus,  to  maintain  the  overall  desired  sample  size  for  activities  of  about  two  per  district,  we  decided 
to  sample  all  activities  offered  by  the  sampled  SAHE  grantees.  Then,  as  we  did  in  our  district  data 
collection,  we  obtained  the  names  of  two  randomly  selected  teachers  who  attended  each  activity.  We 
were  able  to  obtain  teacher  names  for  8 1 SAHE  grantees,  or  88  percent  of  the  SAHE  grantees  in 
which  we  conducted  interviews. 

Response  rates  for  teachers.  Altogether,  we  obtained  the  names  of  334  teachers.  Of  these, 
we  excluded  27  from  the  sampling  frame  because  they  reported  that  they  did  not  attend  the  sampled 
activity  or  because  we  could  not  obtain  a correct  address.  Of  the  remaining  307  teachers,  we 
obtained  completed  responses  from  244,  or  80  percent. 

Sampling  Probabilities  and  Weights  for  the  SAHE-grantee  Project  Director 
Interview 

The  probability  of  selection  for  SAHE-grantee  k (nJk)  can  be  expressed: 


where  n is  the  number  of  SAHE  grantees  in  the  national  sample,  Gk  is  the  grant  size  for  SAHE- 
grantee  k,  and  G is  the  total  grant  size  in  all  SAHE  grantees  in  the  population. 31 

The  sampling  weight  for  SAHE-grantee  k ( wk ) is  the  inverse  of  the  sampling  probability: 

1 G 

w.  = — - = 

.<  nGk 

For  the  district  telephone  interview,  we  weighted  the  results  to  represent  the  number  of 
teachers  in  districts.  For  SAHE  grantees,  we  cannot  carry  out  an  analysis  that  is  exactly  parallel, 
because  SAHE  grantees  are  not  staffed  with  teachers.  Instead,  for  the  SAHE-grantee  analysis,  we 
weighted  the  results  to  represent  the  number  of  Eisenhower-assisted  teacher  participants  in  SAHE 
grantees. 

30  We  did  not  draw  replacements  for  a few  SAHE  grantees  that  we  learned  were  out  of  scope  late  in  the  data 
collection  process,  after  nearly  all  other  interviews  were  complete. 

31  One  IHE/NPO,  with  an  annual  grant  size  of  $749,8 13,  was  selected  with  certainty.  To  compute  the  probabilities 
for  the  remaining  99  non-certainty  IHE/NPOs,  we  set  n=99  and  G=$3 1,778, 388. 

see 
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The  final  SAHE-grantee  weight  is  the  product  of  the  sampling  weight  wk  and  the  number  of 
Eisenhower  participations  in  the  SAHE-grantee  T/:: 
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V 'T’  E ^ 
1 k 


The  first  term  is  a constant.  The  second  term 


te  ^ 

1 k 


is  the  number  of  teacher  participations  per 


dollar.  The  term  reflects  the  fact  that  respondents  in  SAHE  grantees  that  include  more  participating 
teachers  per  grant  dollar  receive  more  weight  in  estimating  results. 

Calculating  the  SAHE-grantee  telephone  interview  weights  requires  an  estimate  for  the 
number  of  teachers  per  dollar  for  each  participating  SAHE  grantee.  Our  SAHE-grantee  data  provide 
two  estimates  of  the  total  number  of  participants,  one  from  the  telephone  interview,  and  the  other 
from  the  activity  lists.  A few  values  for  the  number  of  dollars  per  participant  appeared  to  be  too  high 
or  too  low  to  be  plausible.  Thus,  we  set  a plausible  lower  limit  of  0.2  participants  per  1000  dollars 
and  an  upper  limit  of  1 0 participants  per  1 ,000  dollars.  If  our  estimates  based  on  both  sources  of 
available  data  were  within  these  limits,  we  averaged  the  two  estimates  to  obtain  a final  value.  If  only 
one  was  within  the  limits,  we  used  it.  If  neither  was  within  the  limits,  we  Winsorized  the  value,  by 
selecting  either  the  lower  or  upper  limit,  as  appropriate. 

Sampling  Probabilities  and  Weights  for  the  SAHE-grantee  Teacher  Activity 
Survey 

For  each  SAHE  grantee,  we  sampled  all  of  the  activities  supported  over  the  relevant  period. 
The  probability  of  selection  for  activity  j,  given  that  district  k has  been  selected,  is  thus  one. 

The  probability  of  selection  of  teacher  i,  given  that  activity  j in  SAHE-grantee  k has  been 
selected,  can  be  written: 


Kijk\jk 


where  tjk  is  the  total  number  of  teachers  who  participated  in  activity  j in  SAHE-grantee  k. 

Since  we  selected  all  activities  within  each  sampled  SAHE  grantee,  the  probability  of 
selection  of  activity  j given  that  SAHE-grantee  k has  been  selected  is  one. 

Thus,  the  overall  probability  of  selection  of  teacher  i in  activity  j in  SAHE-grantee  k is: 
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Thus,  the  sampling  weight  for  teacher  i in  activity  j in  SAHE  grantee  is  k (i.e.,  the  inverse  of  the 
sampling  probability)  is: 


wwt  = 
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Computing  the  weight  as  derived  above  would  require  separate  estimates  of 
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JL 
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, the 


number  of  participants  per  grant  dollar,  for  each  activity  j within  each  SAHE-grantee  k.  The 
variation  among  weights  for  different  activities  within  the  same  SAHE  grantee  reflects  fact  that  we 
sampled  all  activities  within  each  SAHE  grantee,  and  thus  teachers  in  large  activities  were  less  likely 
to  be  drawn  into  the  sample  than  teachers  in  small  activities. 


The  activity  lists  provide  data  on  the  number  of  teachers  in  each  activity,  ty,  and  thus  the  lists 
could  in  principle  be  used  to  calculate  distinct  values  for  the  weights  for  each  activity.  But  we 
elected  instead  to  compute  a single  “average”  weight  for  all  teachers  in  the  same  SAHE  grantee.  We 
computed  the  weight  for  SAHE-grantee  k by  summing  the  weights  as  derived  above  for  all  sampled 
teachers  in  all  of  the  activities  SAHE-grantee  k offered  and  then  dividing  by  the  number  of  teachers 
in  the  sample  for  SAHE-grantee  k. 


The  first  part  of  this  computation — the  sum  of  the  weights  across  all  sampled  teachers  in  all 
activities;  in  SAHE-grantee  k,  assuming  2 sampled  teachers  per  activity — is: 
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In  other  words,  the  sum  of  the  teacher  weights  for  all  sampled  teachers  in  SAHE-grantee  k is  the  final 
weight  for  SAHE-grantee  k derived  in  the  telephone  interview  (see  above).  Thus,  the  “average” 
weight  for  each  sampled  teacher  in  SAHE-grantee  k,  which  we  used  for  the  teacher  activity  survey,  is 
the  telephone  interview  weight,  divided  by  the  number  of  teachers  in  the  sample  for  SAHE-grantee  k. 

This  approach  to  computing  the  teacher  weight  for  the  SAHE  grantee  sample  has  two  main 
advantages.  First,  it  makes  the  resulting  teacher  weights  consistent  with  the  telephone  interview 
weights.  In  computing  the  telephone  interview  weights,  as  described  above,  we  averaged  two 


sources  of  data  to  compute  a value  for 


f nr  E ^ 
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— data  from  the  activity  lists  and  data  from  the 


telephone  interviews.  We  have  only  one  source  of  data  on  the  number  of  teachers  in  each  activity — 
the  activity  lists  themselves — and  thus  we  cannot  carry  out  an  analogous  averaging  procedure  for 
each  activity. 


Second,  by  averaging  across  activities,  this  procedure  reduces  the  potential  variation  in 
weights  that  would  arise  due  to  any  errors  in  the  activity  lists — for  example,  missing  teachers  from 
some  activities. 
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Although  we  believe  the  averaging  process  we  chose  has  several  advantages,  we  expect  that 
it  has  relatively  little  effect  on  the  final  results,  in  comparison  to  the  results  that  would  be  obtained 
using  separate  weights  for  each  activity.  In  particular,  48  of  the  92  SAHE  grantees  in  our  sample  had 
only  one  activity,  according  to  the  activity  list  provided,  and  thus  they  were  not  influenced  by  the 
averaging  procedure;  and  many  of  the  rest  had  activities  of  relatively  similar  size. 

Methods  for  SAHE-grantee  Hypothesis  Tests  and  Standard  Error  Estimation 

In  the  sections  below,  we  discuss  our  approach  to  estimating  standard  errors  for  the  SAHE- 
grantee  telephone  interview  and  teacher  activity  results. 

SAHE-grantee  project  director  interviews.  In  the  analyses  of  the  SAHE-grantee  project 
interview  data  reported  in  Chapter  6,  the  significance  tests  are  computed  according  to  the 
assumptions  of  conventional  simple  random  sampling.  These  tests  do  not  take  into  account  the  fact 
that  there  is  some  variation  in  the  weights  across  SAHE  grantees,  due  to  the  variation  in  the  number 
of  teachers  served  per  dollar,  as  discussed  above.  Taking  the  variability  in  the  weights  into  account 
would  produce  standard  errors  somewhat  larger  than  those  upon  which  the  reported  significance  tests 
are  based.  We  have  rerun  the  analyses  taking  the  variability  of  the  weights  into  account,  and  the 
results  of  significance  tests  are  nearly  identical  to  those  reported  in  Chapter  6. 

Teacher  surveys.  Our  analysis  of  the  characteristics  of  Eisenhower-supported  activities 
discussed  in  Chapter  3 rely  on  teacher-reported  data.  The  significance  tests  currently  presented  in 
chapter  3 are  computed  according  to  the  assumptions  of  simple  random  sampling.  More  precise  tests 
can  be  conducted  that  take  into  account  two  factors  not  currently  considered  in  the  reported  results: 
the  fact  that  the  final  weights  vary  across  teachers  in  different  SAHE  grantees  (see  the  discussion 
above),  and  the  fact  that  the  reports  of  teachers  who  attended  the  same  activities  or  attended  activities 
in  the  same  SAHE  grantees  are  not  independent.  Taking  these  factors  into  account  would  produce 
somewhat  larger  standard  errors  than  those  upon  which  the  reported  significance  tests  are  based.  We 
reran  the  analyses  taking  these  factors  into  account,  and  the  significance  test  results  are  very  similar 
to  those  reported  in  Chapter  3. 
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APPENDIX  B 


IN-DEPTH  AND  EXPLORATORY  CASE  STUDIES 


We  conducted  In-depth  Case  Studies  of  10  school  districts  in  five  different  states,  and 
Exploratory  Case  Studies  in  six  districts  in  six  different  states.  The  goal  of  the  case  studies  was  to 
provide  detailed  information  on  how  the  Eisenhower  program  operates  in  selected  states  and  school 
districts.  Further,  the  exploratory  case  studies  were  designed  to  identify  issues  to  help  inform  the 
design  of  survey  instruments  for  the  National  Profile  component  of  the  evaluation.  (For  a report  of 
the  results  of  our  exploratory  case  studies,  see  Birman,  Reeve,  & Sattler,  1998.) 

Selection  of  States,  and  Districts  within  States,  for  In-depth  Case  Studies 

We  used  a two-pronged  approach  to  site  selection  for  the  in-depth  case  studies.  First  we 
selected  states  and  then  districts.  Our  goal  was  to  select  two  districts  located  in  each  of  five  states. 
We  had  three  main  site  selection  criteria:  (1)  demographic  characteristics  of  states  and  districts  and 
geographical  region;  (2)  state-level  education  reform  efforts,  including  the  state’s  role  in 
professional  development;  and  (3)  the  district’s  approach  to  professional  development  and  its 
relationships  with  other  state  and  local  reforms. 

Demographic  Characteristics  and  Geographic  Region 

States  and  districts  vary  in  terms  of  the  size  of  their  populations,  the  demographic 
characteristics  of  their  populations,  and  the  region  of  the  country  in  which  they  are  located.  To 
observe  how  the  Eisenhower  Professional  Development  Program  operates  in  different  contexts,  we 
chose  states  and  districts  to  vary  on  these  factors. 

State  characteristics.  We  selected  five  states  to  represent  different  regions  of  the  country. 
We  chose  one  state  from  each  of  the  following  regions:  the  Midwest,  South,  Southwest,  Pacific 
Northwest,  and  Northeast.  Because  of  the  need  to  achieve  variation  in  terms  of  district  size  and 
demographic  characteristics,  our  sample  of  states  was  weighted  disproportionately  toward  states  with 
relatively  large  populations  (i.e.,  states  with  greater  variety  in  the  characteristics  of  local  districts). 

District  characteristics.  We  considered  the  following  demographic  factors  in  selecting 
districts:  1)  total  enrollment,  2)  racial  and  ethnic  composition  of  the  district’s  students,  3)  poverty 
level  of  the  district’s  students,  and  4)  percent  of  language  minority  students.  These  district-level 
characteristics  are  important  because  we  hypothesized  that  the  Eisenhower  program  operates 
differently  in  districts  of  varying  size  and  because  the  Title  II  program  is  intended  to  serve  the  needs 
of  teachers  of  all  students,  in  particular  those  placed  at  risk.  For  this  reason,  we  selected  more  than 
half  of  the  sample  (i.e.,  six  districts)  with  relatively  high  percentages  of  students  in  poverty,  and  half 
of  the  sample  with  high  percentages  of  student  from  racial  and  ethnic  minority  backgrounds, 
including  two  districts  with  high  percentages  of  students  with  limited  English  proficiency. 

In  addition,  because  larger  districts,  by  definition,  serve  larger  numbers  of  students,  and 
because,  in  general,  we  hypothesized  that  larger  districts  are  more  likely  to  be  involved  in  non- 
traditional  professional  development,  we  selected  a sample  that  overrepresented  districts  with  large 
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numbers  of  students.  Districts  also  were  selected  to  exhibit  variation  in  terms  of  urbanicity  (e.g., 
urban,  suburban,  small  city,  and  rural  districts). 

We  compared  the  national  distribution  of  enrollment,  students  in  poverty,  and  percent 
minorities,  and  chose  our  sample  to  reflect  these  national  distributions.  For  example,  Exhibit  B.l 
shows  how  enrollment  in  our  sample  of  case-study  districts  compares  to  national  figures  on 
enrollment. 


EXHIBIT  B.l 

Number  of  Selected  Districts  by  Student  Enrollment 


District  Enrollment 

Percent  of  All  Children  Served  in 
Districts  of  This  Size 

Number  of  Districts  Selected  for 
Sample 

1-999 

5.8  percent 

1 

1,000-9,999 

44.7 

3 

10,000-24,999 

18.6 

2 

25,000-99,999 

19.3 

2 

100,000+ 

11.6 

2 

Exhibit  B.2  shows  the  enrollment  figures,  the  percent  of  school-aged  children  in  poverty,  and 
the  percent  of  non-white  students  in  each  of  our  case  districts. 


EXHIBIT  B.2 

Demographic  Information  on  the  10  In-depth  Case-study  Districts 


Total  Enrollment 

Percent  School-aged 
Children  in  Poverty 

Percent  Non-white 

Boonetown,  KY 

8,000 

18.5 

9 

Weller,  KY 

5,000 

38.0 

<1 

Richmond,  NY 

9,000 

25.9 

63 

East  City,  NY 

1,000,000 

30.5 

61 

Maple  City,  OH 

64,000 

30.3 

43 

Buckeye,  OH 

11,000 

2.2 

9 

Lone  Star,  TX 

64,000 

56.5 

80 

Rhinestone,  TX 

200,000 

33.6 

88 

Riverside,  WA 

19,000 

3.9 

11 

Rainforest,  WA 

1,500 

20.5 

17 

Selection  of  Exploratory  Case  Districts 

Like  the  in-depth  cases,  we  selected  the  six  exploratory  case  studies,  to  vary  by  region, 
urbanicity,  poverty  level,  and  ethnic  composition.  In  addition  to  being  selected  for  their  demographic 
features,  the  six  exploratory  sites  also  represented  districts  with  features  that  we  believed  might 
influence  the  program’s  implementation.  The  six  sites  were  selected  to  include:  one  district  that 
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participated  in  the  Eisenhower  program  through  a consortium;  at  least  two  sites  that  had  IHE- 
supported  projects  working  in  the  district;  and  two  districts  located  in  states  that  received  waivers 
from  ED  allowing  greater  proportions  of  Eisenhower  funds  to  support  professional  development  in 
areas  outside  mathematics  and  science.  Waivers  for  individual  states  from  particular  legislative 
requirements  for  the  use  of  Title  II  money  can  have  important  consequences  for  how  states  and 
districts  use  those  funds,  as  can  Ed-Flex  status,  which  allows  selected  states  to  grant  waivers  to 
individual  districts. 

Features  of  State  Educational  Reform  and  Approach  to  Professional  Development 

We  selected  states  that  varied  in  their  statewide  education  system  and  reform  approaches,  as 
well  as  their  approaches  to  professional  development.  Features  of  the  state’s  educational  system  and 
reform  efforts  form  important  contextual  background  for  the  implementation  of  the  Eisenhower 
Professional  Development  Program.  For  example,  the  adoption  of  content  and  performance 
standards  or  a new  assessment  system  can  have  dramatic  implications  for  how  states  and  districts 
direct  their  professional  development  priorities.  Similarly,  the  presence  of  other  educational 
initiatives,  such  as  National  Science  Foundation-funded  State  Systemic  Initiatives  (SSI),  are  often 
integrally  linked  with  the  Eisenhower  program  in  states  and  districts. 

States  also  vary  significantly  in  the  role  they  play  in  guiding  and  providing  professional 
development.  Some  states  play  virtually  no  role,  leaving  professional  development  almost  entirely 
up  to  the  discretion  of  local  districts.  In  many  states,  the  state’s  most  active  involvement  in 
professional  development  is  indirect;  the  state  expresses  its  requirements  for  individual  teachers’ 
professional  development  through  its  requirements  for  certification  and  recertification.  Other  states, 
however,  take  much  more  active  roles  in  professional  development,  often  by  setting  standards  for 
high-quality  professional  development,  mandating  that  districts,  schools,  and  teachers  develop 
professional  development  plans,  and  by  directing  funds  to  districts  expressly  to  support  professional 
development. 

In  the  sample  of  five  states  for  the  in-depth  case  studies,  we  chose  states  to  have  a range  of 
approaches  to  reform  and  the  extent  of  state  involvement  in  professional  development: 

• Kentucky  has  implemented  comprehensive  school  reform  efforts.  The  Kentucky 
Education  Reform  Act  of  1990  (KERA)  introduced  educational  goals,  an  assessment 
system,  increased  funding  for  professional  development,  a new  credentialing  system, 
early  childhood  programs,  technology,  ’’full-service  schools,”  changes  in  governance 
structure,  and  an  equalized  funding  system.  Professional  development  is  a key 
component  of  KERA,  which  provides  funding  for  professional  development  ($23  per 
average  daily  attendance)  and  requires  that  responsibility  for  65  percent  of  professional 
development  funds  be  held  by  school-based  councils. 

• New  York’s  content  and  performance  standards  (with  the  exception  of  social  studies 
standards,  which  are  considered  weak)  are  highly  regarded.  Although  New  York  has 
traditionally  been  considered  to  lag  in  professional  development  at  the  state  level,  the 
soon-to-be-implemented  requirement  that  all  students  take  the  Regents’  examinations  has 
introduced  great  challenges  for  the  state’s  teaching  force. 

• Ohio  is  in  the  process  of  merging  content  standards  with  new  performance  standards  for 
schools  and  districts,  as  well  as  revising  its  teaching  standards  and  introducing  a 
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mentoring  program  for  all  beginning  teachers.  Ohio’s  content  standards  in  mathematics 
were  rated  “exemplary”  by  the  AFT  in  its  review  of  state  standards.  In  addition,  several 
districts  in  Ohio  are  considered  to  be  strong  in  their  professional  development  programs. 

• Texas  has  a strong  system  of  holding  schools  and  districts  accountable  for  student 
performance,  has  raised  expectations  for  student  performance,  and  recently  has  rewritten 
its  education  code  to  scale  back  the  role  of  state  government  in  schools.  During  the 
1990s,  Texas  mandated  that  all  beginning  teachers  be  assigned  a mentor  and  that 
professional  development  be  primarily  school-based  and  designed  to  support  the 
achievement  of  schools’  performance  objectives.  Achievement  in  the  state  has  risen  in 
recent  years;  and  several  districts  have  been  noted  for  their  strong  professional 
development  programs. 

• Washington  focuses  on  content  standards  in  their  Essential  Academic  Learning  reform. 
The  state  emphasizes  school-based  professional  development;  the  state  also  channels 
teacher  certification  fees  back  to  support  preservice  and  in-service  training.  The  state 
awarded  districts  $30  million  in  S.tudent  Learning  Improvement  Grants  in  1995-96,  and 
about  75  percent  of  that  money  was  used  to  support  professional  development. 

Features  of  District  Approach  to  Professional  Development  and  Education  Reforms 

Even  in  states  that  are  relatively  highly  involved  in  professional  development,  it  remains 
primarily  a local  responsibility.  Districts  may  take  varying  approaches  to  teachers’  professional 
development.  In  some  districts,  professional  development  may  remain  centered  on  short-term 
“traditional”  workshops,  seminars,  and  conferences.  Other  districts  may  be  moving  toward 
professional  development  with  more  activities  that  are  sustained  over  time  and  that  allow  teachers 
opportunities  to  practice  new  techniques  in  their  classrooms  and  receive  feedback  from  colleagues 
and  professional  development  providers.  Still  other  districts  may  focus  their  professional 
development  resources  on  yet  more  “reform”  types  of  professional  development,  such  as  teacher 
networks  or  teacher  mentoring  programs.  We  sought  districts  that  were  engaging  in  professional 
development  consistent  with  at  least  several  dimensions  of  high-quality  professional  development, 
such  as  extended  in-depth  learning  opportunities,  efforts  to  build  a learning  community  of  science 
and  mathematics  teachers,  linking  professional  development  with  other  state  and  district  reforms,  and 
use  of  evaluation  and  other  types  of  continuous  improvement  efforts. 

The  sample  of  10  districts  was  chosen  to  include  examples  of  these  and  other  types  of 
approaches  to  professional  development.  We  also  chose  districts  so  that  our  sample  included  districts 
with  a range  of  different  local  reform  efforts,  such  as  NSF’s  Urban  Systemic  Initiatives  or  Local 
Systemic  Change  projects. 

We  selected  districts  that  seemed  to  emphasize  reform  types  of  high-quality  approaches  to 
professional  development.  The  approach  that  led  us  to  select  a district  could  be  the  district’s  overall 
plan  for  professional  development,  a piece  of  its  approach,  or  even  a single  program.  Typically  we 
would  not  expect  the  approach  to  be  the  district’s  only  approach  to  professional  development,  but  we 
would  want  the  district  to  give  it  enough  emphasis  to  be  a significant  component  of  the  district’s 
professional  development  program.  We  attempted  to  achieve  a range  across  the  sample  of  10 
districts  in  the  types  of  approaches  they  employ  (e.g.,  a district  that  emphasizes  mentoring,  a district 
whose  teachers  all  have  individual  professional  development  plans,  a district  in  which  professional 
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development  is  generally  developed  by  teachers).  See  Chapter  3 for  a detailed  description  of  high- 
quality  characteristics  of  professional  development. 

Exhibit  B.3  provides  examples  of  each  case  study  district’s  Eisenhower-assisted  professional 
development  activities  and  demonstrates  the  variety  of  professional  development  types  of  activities 
in  the  districts.  Exhibit  B.4  provides  similar  information  for  our  exploratory  case  sites. 

EXHIBIT  B.3 


In-depth  Case  Studies 


District  Pseudonym 

Description  of  Eisenhower-assisted  Activities 

Richmond,  New  York 

• School-based,  2-day  workshop  focusing  on  “hands-on  math.” 

• Curriculum  review  committees,  workshops,  and  follow-up  study  group  activities  relating 
to  state  assessment  and  core  content  in  math. 

• Resource  teachers  coach  teachers  needing  assistance  with  lesson  development  and 
delivery. 

East  City,  New  York 

• In-district  workshops  and  follow-up  mentoring  activities  focus  on  a variety  of  math  and 
science  topics,  such  as  expanding  the  pilot  math  program,  Connected  Math,  with  the 
option  to  request  in-classroom  assistance  as  needed. 

• Ongoing  immersion  activities  related  to  projects  in  conjunction  with  Botanical  Gardens, 
the  Bronx  Zoo,  and  other  community  organizations. 

• T uition  reimbursement  is  available  for  college  courses. 

Maple  City,  Ohio 

• Lead  teachers  attend  extended  workshops  to  develop  integrated,  real-life,  thematic  units 
for  elementary  and  middle  school  mathematics  and  science  instruction. 

• Selected  teachers  attend  10,  week-long  summer  workshops  dedicated  to  implementing 
model  math  and  science  programs  such  as  Connecting  Math  and  Science,  Building 
Bridges,  and  Hands-on  Math  and  Science. 

• School-based  workshops  surround  the  implementation  of  “Best  Practices”  strategies. 

Buckeye,  Ohio 

• Lead  teachers  conduct  hands-on  inservice  that  supports  an  integrated  approach  to 
instruction  and  coach  teachers  when  implementing  new  practices  through  networking, 
team-teaching,  and  mentoring. 

• Tuition  reimbursement  for  out-of-district  conferences,  college  courses,  day-long, 
weekend,  and  summer  workshops. 

Rhinestone,  Texas 

• Master  teachers  provide  materials,  deliver  workshops  using  manipulatives,  and  provide 
mentoring  support  for  the  implementation  of  integrated  math  activities. 

• Institute  focuses  on  improvement  of  algebra  lesson  preparation  aligned  with  state 
assessment  and  curriculum  and  alignment  review  committees  ensure  articulation  to 
state  standards  across  grade  levels. 

Lone  Star,  Texas 

• During  the  year  of  our  case  visits,  Lone  Star,  Texas  school  district  was  undergoing 
major  reorganization  and  as  a result  did  not  offer  any  Eisenhower-assisted  professional 
development  activities. 

EXHIBIT  B.3  (Continued) 
In-depth  Case  Studies 


District  Pseudonym 

DESCRIPTION  OF  ElSENHOWER-ASSISTED  ACTIVITIES 

Riverside, 

Washington 

• Task  force  develops  science  kits;  committees  for  each  grade  level  determine  principles, 
activities,  materials,  and  pilot  and  implement  kits. 

• Summer  Science  Institute  and  fellowship  opportunity  is  held  in  conjunction  with  local 
universities  and  businesses. 

• Teachers  receive  professional  organization  membership  for  journals  and  resources. 

• Short  workshops  focus  on  instruction,  and  support  for  conferences  and  other  networking 
opportunities  are  available  as  needed. 

Rainforest, 

Washington 

• In-district,  week-long  science  institute  focuses  on  hands-on  activities,  use  of  manipulatives, 
lesson  demonstration,  and  instructional  strategies.  Follow-up  includes  one-on-one  mentoring 
sessions,  teacher  observations,  and  teacher  study  groups. 

• Partnerships  in  Education  program  allows  teachers  to  network  with  state  and  federal 
agencies  research  on  potential  student  projects  and  field  trips  in  science. 

Weller,  Kentucky 

• Goals  2000  master  teachers  give  lesson  demonstrations  and  coach  other  teachers  with  unit 
development. 

• In-district  workshops  are  dedicated  to  teacher  practice  with  graphing  calculators  for  state 
assessment,  new  math  requirements,  state  core  content,  and  planning  for  the  development 
of  accountability  grades  for  math  and  science. 

• Hands-on  inservice  focuses  on  higher  order  thinking  skills  and  “good  practices”  and  supplies 
applicable  materials  or  kits. 

• Curriculum  task  force  provides  curriculum  development  and  alignment  workshops. 

• Master  teachers  observe  and  coach  new  teachers  in  the  district. 

Boonetown, 

Kentucky 

• Resource  teachers  assigned  to  two  schools  model  class  lessons,  observe  in  classrooms, 
and  coach  individual  teachers. 

• In-district  workshops  focus  on  improvement  of  district  test  scores  in  math. 

• Math  teachers  work  on  curriculum  review  and  alignment  committees. 

• Tuition  reimbursement  available  for  teachers  of  all  grade  levels. 

West  City,  California 

• Lead  teachers  are  selected  to  develop  a 3-day,  district-wide  staff  development  program, 
weekend  seminars,  mentoring,  and  follow-up  activities. 

• Curriculum  committees  work  to  approve  adoption  of  new  curricula. 

• Study  groups  share  and  compare  curriculum  and  instruction  strategies. 

• T uition  reimbursements,  stipends,  and  substitutes  are  available  for  time  spent  at  district 
conferences,  courses,  and  workshops. 
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EXHIBIT  B.4 


Exploratory  Case  Districts 

District  Pseudonym 

DESCRIPTION  OF  EISENHOWER-ASSISTED  ACTIVITIES 

Middle  City,  Wisconsin 

• Lead  teachers  attend  year-long  institute  for  framework  construction  in  math  and  science. 

• Curriculum  committee  plans  adoption  of  new  curricula. 

• In-district  workshops  decided  by  committee,  focus  on  comprehensive  standards, 
curriculum,  and  assessment,  and  include  follow-up  mini-grants  for  teacher’s  individual 
research  projects. 

• Teacher  task  forces  and  study  groups  focus  on  curriculum-defining  activities. 

• Institutes,  mentoring,  peer  coaching  focus  on  the  improvement  of  math  and  science 
teaching  strategies  and  inclusion  of  multicultural  science  education. 

• Internships  offer  teachers  summer  work  experiences  in  business  and  industry,  in  order  to 
write  “real  life"  curriculum  units. 

South  City,  Florida 

• In-district  institutes  and  out-of  district  conferences  focus  on  curriculum  content  and 
teaching  methods. 

• Mentors  lead  activities  to  help  math  and  science  teachers  improve  instruction  including 
follow-up  coaching  in  math  and  science. 

• Mini-grants  support  inservice  presentations  and  the  purchase  of  classroom  materials. 

• T uition  reimbursement  is  available  for  college  and  university  programs. 

Commuteville,  Virginia 

• Colleague  teacher  program  focuses  on  needs  of  first-year  teachers. 

• Lead  teachers  attend  30-week  institute  for  leadership  development  in  math  and  science 
to  coach  other  teachers  when  developing  concepts  and  integrating  technology. 

• District  operates  a professional  development  “academy,”  including  five-week  math  and 
science  workshops  and  institutes  used  primarily  for  recertification. 

• Tuition  reimbursement  available  for  graduate  courses  and  out-of-district  conferences,  and 
support  for  materials. 

Northtown,  Connecticut 

• In-district  workshops  involving  presentations  and  discussion  focus  on  cooperative 
learning  in  math  and  science,  and  the  creation  of  hands-on  learning  kits  for  elementary 
students. 

• Teacher  representatives  and  lead  teachers  attend  out-of-district  conferences  and 
institutes  focusing  on  “best  practices,”  such  as  portfolio  assessment,  and  work  in 
networks  and  study  groups  to  share  information  with  fellow  faculty. 

• School-based  facilitators  coach  and  support  elementary  school  teachers. 

• T uition  reimbursement  available  for  out-of-district  conferences,  workshops,  and  institutes. 

Countryplace,  Missouri 

• District  integrates  Eisenhower  out-of-district  conferences,  workshops,  and  inservices  on 
math  and  science  with  state-level  reform  program  and  professional  development 
committee  goals  focusing  on  school  improvement. 

• Informal  study  groups  discuss  issues  before  and  after  conferences. 

• Some  tuition  reimbursement  is  available  for  out-of  district  conferences. 

Selection  Process 

In  selecting  sites  for  the  10  in-depth  case  studies,  we  drew  primarily  upon  three  types  of 
resources.  First,  we  reviewed  documents  pertinent  to  states’  educational  systems  and  professional 
development  efforts;  for  example,  we  drew  heavily  upon  the  Consortium  for  Policy  Research  in 
Education  (CPRE)  report,  Policies  and  Programs  for  Professional  Development  of  Teachers: 

Profiles  of  the  States.  Second,  we  discussed  both  states  and  districts  with  a team  of  outside  experts  in 
the  areas  of  education  reform  and  professional  development.  We  consulted  the  following  experts: 

• Susan  Fuhrman  of  CPRE, 

• Jane  David  of  Bay  Area  Research  Group, 

• Katie  Haycock  of  the  Education  Trust, 

• Marsha  Levine  of  George  Washington  University, 

• Barbara  Scott  Nelson  of  the  Education  Development  Corporation, 

• Barbara  Neufeld  of  Education  Matters, 

• Michael  Knapp  of  the  University  of  Washington, 

• Jan  Kettelwell  of  the  University  of  Georgia,  and 

• Members  of  our  design  team: 

• Mary  Kennedy,  Michigan  State  University, 

• Susan  Loucks-Horsley,  WestEd,  National  Academy  of  Sciences 

• Jennifer  O’Day,  University  of  Wisconsin,  Madison, 

• Senta  Raizen,  The  National  Center  for  Improving  Science  Education,  WestEd, 

• Thomas  Romberg,  Wisconsin  Center  for  Education  Research,  and 

• Norman  Webb,  Wisconsin  Center  for  Education  Research. 

Finally,  in  collecting  information  about  potential  district  case  sites,  we  received  information  from  the 
state  Eisenhower  coordinators  in  many  states. 

Data  Collection  Procedures 

Exploratory  Case  Studies 

In  April  and  May  of  1997,  we  conducted  focus  groups  of  teacher-participants  in  Eisenhower- 
assisted  activities  and  professional  development  providers  of  Eisenhower- assisted  activities,  in  each 
of  the  six  exploratory  case  districts.  Each  focus  group  included  approximately  10  participants,  chosen 
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to  reflect  a range  of  .variation  of  experiences  and  perspectives.  In  the  focus  groups,  we  asked 
questions  about:  1 ) school  organization  and  decision  making;  2)  curriculum  and  instruction;  3) 
characteristics  of  professional  development  activities;  4)  teachers’  and  principals’  goals  for 
professional  development,  and  how  they  learn  about  activities  and  choose  among  opportunities;  5) 
links  among  different  professional  development  activities  and  between  professional  development 
activities  and  other  reforms;  6)  ways  the  school  supports  or  constrains  teachers’  ability  to  take 
advantage  of  professional  development  opportunities;  and  7)  perceptions  of  the  effectiveness  and 
usefulness  of  the  activities. 

In  addition,  we  conducted  semi-structured  interviews  with  district-level  staff  in  each  of  the 
districts.  In  each  of  the  six  exploratory  case  districts,  we  conducted  interviews  with  the 
superintendent,  the  Eisenhower  coordinators,  other  professional  development  coordinators  in  the 
district,  and  the  mathematics  and  science  coordinators. 

In-depth  Case  Studies 

We  conducted  our  in-depth  case  studies  from  January  through  June  of  1998.  We  spent  a total 
of  one  week  (i.e.,  approximately  five  to  seven  work  days)  in  each  of  the  10  selected  districts.  Two 
trained  interviewers  from  our  research  team  went  to  each  site.  We  spent  about  one  day  conducting 
interviews  at  the  state  level,  between  a half  a day  and  a day  conducting  district-level  interviews,  and 
between  three  and  four  days  interviewing  and  observing  teachers  and  principals  at  schools  in  the 
district.  These  interviews  and  observations  at  three  schools  in  each  of  our  case-study  districts  were 
part  of  the  Longitudinal  Study  of  Teacher  Change.  (For  details  on  the  design  and  methodology  of 
this  component  of  the  evaluation,  see  Appendix  C;  for  the  results  of  this  part  of  the  study,  see 
Chapter  2.) 

For  the  in-depth  case  studies,  we  conducted  approximately  six  semi-structured  interviews 
with  state-level  staff  in  each  state,  including  the  Eisenhower  SEA  and  SAHE  coordinators,  and 
professional  development,  curriculum,  Title  I,  and  evaluation  coordinators.  We  asked  state-level 
coordinators  questions  about  the  following  topics:  1)  the  state’s  organizational  structure  as  it  pertains 
to  the  education  department;  2)  the  state’s  professional  development  efforts,  and  how  the  Eisenhower 
program  fits  into  those  efforts;  3)  systemic  reform  efforts  that  are  occurring  in  the  state;  4)  practices 
and  policies  for  targeting  and  recruiting  teachers  for  professional  development;  and  5)  the  state 
processes  and  procedures  for  continuous  improvement  such  as  assessing  professional  development 
needs,  and  planning,  evaluating,  and  monitoring  efforts. 

In  addition  to  the  state-level  interviews,  we  also  conducted  about  four  or  five  interviews  in 
each  of  the  10  case  districts  with  district-level  staff,  including  the  district  Eisenhower  coordinator, 
the  coordinator  of  activities  sponsored  by  institutions  for  higher  education,  if  such  activities  existed 
in  the  district,  and  the  professional  development,  curriculum,  and  Title  I coordinators.  Thus,  we 
conducted  about  30  state-level  interviews  and  43  district-level  interviews  in  all  5 in-depth  case  study 
states  combined. 

In  the  district-level  interviews,  we  asked  questions  about  how  decisions  were  made  about  the 
use  of  Title  II  funds,  and  the  nature  and  extent  of  the  coordination  and  alignment  of  the  Eisenhower- 
assisted  activities  with  other  professional  development  and  education  reform  efforts.  Interviews  also 
addressed  the  following  topics:  1)  the  district  structure  and  how  Eisenhower  funds  are  administered 
within  that  structure;  2)  the  administration  of  the  Eisenhower-assisted  activities  program,  and  how 
the  district’s  Eisenhower  program  fits  into  state  and  district  reform  efforts;  3)  the  relationship 
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between  the  Eisenhower-assisted  activities  and  local  or  state  systemic  reform  efforts;  4)  targeting  and 
recruiting  teachers  to  participate  in  Eisenhower-assisted  professional  development  activities;  and  5) 
continuous  improvement  efforts,  such  as  needs,  assessments,  planning,  guidance,  technical 
assistance,  monitoring,  and  reporting. 

In  addition,  for  each  case  district,  interviewers  collected  and  reviewed  state  and  district 
documents,  such  as  curriculum  standards  and  assessments,  as  well  as  extensive  documentation  on  the 
professional  development  activities  that  the  district  provided,  including  examples  of  curricula  for 
activities,  and  schedules  and  syllabi  for  the  activities. 
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APPENDIX  C 


LONGITUDINAL  STUDY  OF  TEACHER  CHANGE 


In  the  Longitudinal  Study  of  Teacher  Change , we  collected  data  from  teachers  in  30 
schools — three  schools  in  each  of  our  ten  In-Depth  case  districts.  We  surveyed  teachers  at  three  time 
points:  in  the  fall  of  1997,  the  spring  of  1998,  and  the  spring  of  1999.  Chapter  2 reports  results  from 
analyses  of  the  baseline  wave  of  the  survey,  conducted  in  the  fall  of  1997. 

In  the  baseline  wave,  we  asked  teachers  to  describe  a mathematics  or  science  course  they 
taught  during  the  1996-97  school  year.  In  the  sections  below,  we  discuss  the  sample  design  and 
response.  We  then  discuss  the  sample  of  courses  the  teachers  described  in  the  surveys. 


Sample  Design  and  Response  Rates  for  the  Longitudinal 
Teacher  Survey  Baseline  Wave  (Fall  1997) 

Designing  the  Sample 

In  selecting  the  overall  sample  of  30  schools,  we  balanced  several  objectives.  First,  we  sought 
schools  in  which  teachers  were  likely  to  participate  in  Eisenhower- assisted  activities  over  the  1997- 
98  year,  the  year  in  which  we  conducted  site  visits,  as  described  in  Appendix  B.  In  addition,  we 
desired  a mix  of  elementary,  middle,  and  high  schools.  Finally,  we  sought  schools  that  varied  in 
demographic  composition,  including  percent  of  students  in  poverty,  as  measured  by  eligibility  for 
free  lunch,  and  percent  minority.  In  particular,  we  planned  to  oversample  high-poverty  schools, 
those  with  over  50  percent  of  students  eligible  for  free  lunch,  because  the  Eisenhower  program 
intends  to  encourage  the  participation  of  teachers  in  such  schools. 

To  achieve  these  objectives,  we  asked  the  Eisenhower  coordinators  in  each  case  district  to 
help  us  identify  schools  that  met  our  criteria,  and  that  were  willing  to  participate  in  our  site  visits, 
longitudinal  survey,  and  classroom  observations.  Based  on  information  provided  by  the 
coordinators,  as  well  as  demographic  information  from  the  Common  Core  of  Data,  we  selected  one 
elementary,  one  middle,  and  one  high  school  in  each  in-depth  case  district.1 

Of  the  30  schools  in  our  final  sample,  15  are  high  poverty  and  12  have  more  than  50  percent 
minority  students.  In  the  fall  of  1997,  student  enrollment  in  the  schools  ranged  from  247  students  in 
one  rural  elementary  school  to  1,554  students  in  a suburban  high  school,  with  an  average  size  of  818 
students. 

In  the  30  schools,  we  planned  to  survey  all  teachers  who  taught  mathematics  or  science 
during  the  1997-98  year.  (We  did  not  attempt  to  locate  teachers  who  taught  mathematics  or  science 
in  the  sample  schools  in  1996-97  but  were  no  longer  teaching  in  the  same  school  in  1997-98.)  To 
identify  teachers  meeting  our  criteria,  we  asked  the  principals  in  each  of  the  30  schools  to  provide  a 
roster  of  all  teachers,  indicating  whether  each  teacher  taught  mathematics,  science,  or  both. 


1 One  district.  East  City,  includes  only  elementary  and  middle  schools,  so  we  chose  two  elementary  and  one  middle. 
In  a second  district,  Weller,  two  K-8  schools  were  selected  rather  than  an  elementary  and  a middle  school. 
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In  most  cases,  the  elementary  schools  in  our  sample  organized  instruction  in  self-contained 
classrooms,  and  thus  most  teachers  were  general  elementary  teachers  who  taught  all  subjects 
including  both  science  and  mathematics.  In  schools  with  self-contained  classrooms,  we  included  all 
elementary  teachers  in  our  sample.  We  randomly  selected  half  of  the  elementary  teachers  to  receive 
a mathematics  version  of  the  survey  and  half  to  receive  a science  version.  Some  elementary  schools 
had  specialist  teachers  who  taught  only  mathematics  or  science  and  we  also  included  these  teachers 
in  our  sample. 

Almost  all  of  the  middle  and  high  schools  in  our  sample  were  departmentalized,  with 
teachers  who  were  subject-matter  specialists.  We  included  all  teafchers  identified  as  mathematics  or 
science  teachers  in  departmentalized  schools,  along  with  any  other  teachers  the  schools  identified  as 
teaching  some  mathematics  or  science,  such  as  the  special  education  teacher. 

Response  Rates  for  Teachers 

Across  the  30  schools,  a total  of  575  teachers  were  asked  to  complete  the  baseline  wave  of  the 
survey.  Of  the  575  teachers  surveyed,  430  returned  a completed  survey,  for  a response  rate  of  75 
percent  (see  Exhibit  C.l).  There  are  differences  in  response  rates  by  school;  in  many  schools, 
response  rates  are  nearly  100  percent,  and  in  others  they  are  below  70  percent.  Of  the  responding 
teachers,  42  reported  that  they  did  not  teach  mathematics  or  science  in  the  1996-97  school  year. 
Generally,  these  42  teachers  were  new  teachers  in  1997,  or  had  not  taught  mathematics  or  science 
before  1997.  Teachers  who  did  not  teach  mathematics  or  science  were  instructed  to  skip  the  subject- 
specific  sections  of  their  surveys.  Thus,  the  sample  of  teachers  describing  their  mathematics  or 
science  instruction  for  1996-97  consists  of  388  teachers. 

Of  these  388  teachers,  355  (91  percent)  completed  a sufficiently  high  proportion  of  the 
classroom  content  items  on  the  survey  to  include  their  responses  in  the  analyses  of  content  discussed 
in  Chapter  2,  and  383  (99  percent)  completed  the  necessary  items  on  pedagogy2  (see  Exhibit  C.l).  In 
order  to  examine  the  relationship  between  content  and  pedagogy,  we  limited  the  sample  for  the 
analyses  of  pedagogy  reported  in  Chapter  2 to  the  350  teachers  who  completed  both  the  content  and 
pedagogy  sections. 


2 Almost  half  of  the  teachers  who  did  not  fully  complete  the  content  items  were  elementary  teachers  who  received 
the  science  version  of  the  survey.  It  is  possible  that  these  teachers  did  not  believe  they  taught  science  with  sufficient 
frequency  or  with  sufficient  depth  to  complete  the  content  section. 


EXHIBIT  C.1 


Response  Rates 

Baseline  Longitudinal  Teacher  Survey,  Fall  1997 


Mathematics 

Science 

Elementary 

School 

Middle 

School 

High 

School 

Elementary 

School 

Middle 

School 

High 

School 

TOTAL 

A.  Number  of  teachers 
sent  surveys 

137 

65 

89 

133 

64 

87 

575 

B.  Number  of  surveys 
returned 

91 

43 

73 

101 

48 

74 

430 

C.  Survey  response  rate 
(row  B divided  by  row 
A) 

66% 

66% 

82% 

76% 

75% 

85% 

75% 

D.  Number  of  responding 
teachers  who  did  not 
teach  mathematics  or 
science  during  1996- 
97  year 

12 

3 

0 

16 

5 

6 

42 

E.  Number  of  responding 
teachers  who  taught 
mathematics  or 
science  during  1996- 
97  year  (row  B minus 
row  D) 

79 

40 

73 

85 

43 

68 

388 

F.  Number  of  teachers 
who  completed  survey 
but  provided 
inadequate  data  for 
analysis  of  content 
taught 

5 

2 

4 

16 

2 

4 

33 

G.  Percent  of  teachers 
completing  content 
items  (row  E minus 
row  F divided  by  row 

E) 

94% 

95% 

95% 

81% 

95% 

94% 

91% 

H.  Number  of  teachers 
who  completed  survey 
but  provided 
inadequate  data  for 
analysis  of  pedagogy 

2 

0 

1 

1 

0 

1 

5 

1.  Percent  of  teachers 
completing  pedagogy 
items  (row  E minus 
row  H divided  by  row 

E) 

97% 

100% 

99% 

99% 

100% 

99% 

99% 

J.  Percent  of  teachers 
completing  both 
content  and  pedagogy 
items 

91% 

95% 

93% 

80% 

95% 

93% 

90% 
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Sample  of  Teachers 


The  teachers  in  the  final  analysis  sample  are  fairly  representative  of  the  general  teaching 
population  (see  Exhibit  C.2).  Nationally,  73  percent  of  teachers  are  female  (Snyder  et  al.,  1999);  in 
this  sample,  74  percent  are  female.  Fourteen  percent  of  all  teachers  are  minorities  (Snyder  et  al., 
1999)  and  18  percent  of  this  sample  are  minorities.  Almost  10  percent  of  all  teachers  have  less  than 
three  years  of  teaching  experience  (Snyder  et  al.,  1999);  in  the  Eisenhower  Longitudinal  Study 
sample,  9 percent  of  teachers  have  less  than  three  years  of  teaching  experience  in  total,  and  1 1 
percent  of  teachers  have  less  than  three  years  of  teaching  experience  in  the  surveyed  subject. 

EXHIBIT  C.2 

Sample  of  Teachers:  Demographics  Baseline  Longitudinal  Teacher  Survey, 

Fall  1997 

Percent  (Number3) 


Mathematics  I Science 


. ' J * ' . :j  S • 

Elementary 
; School  Si 

& Middle 
•)  School 

High  i 
Schools 

Elementary  | 
School  :i 

: Middle 
School 

High 

School 

TOTAL 

| Gender  | 

Female 

95  (70) 

77  (27) 

57  (39) 

96  (65) 

65  (26) 

51  (32) 

74  (259) 

Male 

5 (4) 

23  (8) 

43  (29) 

4 (3) 

35(14) 

49  (31) 

26  (89) 

Total 

100  (74) 

100  (35) 

100  (68) 

100  (68) 

100  (40) 

100  (63) 

100  (348) 

| Ethnicity/Race 

Asian  or  Pacific 
Islander 

2 (1) 

" 

2 (1) 

1 (2) 

African 

American 

8 (6) 

9 (3) 

3 (2) 

3 (2) 

13  (5) 

10  (6) 

7 (24) 

White 

75  (54) 

. 77127) 

93  (63) 

78  (53) 

83  (33) 

83  (52) 

82  (282) 

Hispanic 

17(12) 

14  (51 

2 (D 

16  01) 

5 (2) 

5 (3L_ 

10  (34) 

Other 

- 

2 (1) 

3 (2) 

- 

2 (D 

1 (4) 

Total 

100  (72) 

100  (35) 

100  (68) 

100  (68) 

100(40) 

100  (63) 

100  (346) 

Novice  Teachers" 

Across  subjects 

13  (9) 

7 (2) 

3 (2) 

11  (7)  . 

6 (2) 

12  (7) 

9 (29) 

In  sun/eyed  subject 

11  (6) 

10  (31 

§ (3) 

16  (8) 

12  (4) 

14  (8) 

11  (32) 

"Three  or  fewer  years  of  teaching. 


Sample  of  Courses  Described  in  the  Longitudinal  Teacher  Survey,  Baseline 
Wave  (Fall  1997) 

In  the  longitudinal  teacher  survey,  we  asked  teachers  to  select  a mathematics  or  science 
course  to  describe,  choosing,  if  possible,  a year-long  course  they  had  taught  in  1996-97,  were 
continuing  to  teach  in  1997-98,  and  expected  to  teach  in  1998-99.  If  there  were  several  courses  they 
could  choose,  teachers  were  asked  to  select  a course  that  enrolled  students  performing  at  mixed 
achievement  levels. 


3 Because  some  teachers  did  not  complete  demographic  information  on  the  surveys,  the  numbers  of  teachers 
reported  here  are  slightly  lower  than  the  numbers  reported  in  the  analyses. 
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In  general,  teachers  described  year-long,  mixed  achievement  courses  (see  Exhibit  C.3). 
Although  most  teachers  described  year-long  courses,  some  teachers,  especially  science  and  high 
school  teachers,  focused  on  semester  courses.  Most  teachers  chose  courses  enrolling  students  of 
mixed  achievement  levels,  but  some  teachers,  especially  mathematics  teachers  and  high  school 
teachers,  described  classes  enrolling  students  of  homogeneous  low  or  high  achievement. 

EXHIBIT  C.3 

Sample  of  Courses  Described  in  the  Baseline  Longitudinal  Teacher  Survey, 

Falll  997  (n=355*) 


Mathematics 

Science 

Elementary 

School 

Middle 

School 

High 

School 

Elementary 

School 

Middle 

School 

High 

School 

TOTAL 

| Duration 

Year 

92% 

97% 

74% 

73% 

87% 

73% 

81% 

Semester 

6% 

0% 

26% 

15% 

3% 

24% 

14% 

Other 

2% 

3% 

0% 

12% 

10% 

3% 

5% 

Total 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

Composition 

Homogeneous  high 

1% 

11% 

16% 

0% 

2% 

14% 

7% 

Homogeneous  middle 

9% 

18% 

32% 

3% 

15% 

23% 

17% 

Homogeneous  low 

9% 

16% 

17% 

6% 

% 

8% 

10% 

Mixed 

80% 

53% 

35% 

91% 

78% 

55% 

66% 

Total 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

* In  some  cases,  the  n is  slightly  lower  due  to  missing  data. 


We  asked  teachers  to  report  the  title  of  the  course  they  described,  as  well  as  the  typical  grade 
levels  of  students  enrolled.  Not  surprisingly,  elementary  school  teachers  generally  described 
mathematics  and  science  instruction  in  self-contained  classrooms  enrolling  students  at  one  or  two 
grade  levels  (e.g.,  “4,h'grade  mathematics”).  Most  middle  school  teachers  described  courses  titled 
“7,h'grade  or  8,h'grade  mathematics,”  but  10  of  the  middle  school  mathematics  teachers  and  eight  of 
the  middle  school  science  teachers  described  courses  with  more  specific  titles  indicating  the  focus  of 
the  course  (i.e.,  pre-algebra  and  algebra  for  mathematics,  life,  earth,  and  physical  science). 

Most  of  the  high  school  teachers  described  courses  with  specific  titles.  Of  the  high  school 
mathematics  teachers  surveyed,  32  described  algebra  courses,  13  described  geometry  courses,  and  a 
few  each  described  calculus,  integrated  math,  or  trigonometry.  Of  the  high  school  science  teachers, 
18  described  biology  courses  (with  a few  honors  biology),  13  described  chemistry  courses,  1 1 
described  physics  courses,  and  one  to  three  teachers  described  earth  science,  physical  science,  or 
astronomy.  Four  of  the  science  courses  and  five  of  the  mathematics  courses  were  honors,  advanced, 
or  advanced  placement  courses. 
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APPENDIX  D 


SUPPLEMENT  FOR  CHAPTER  2 


In  this  appendix,  we  describe  the  methods  used  to  analyze  the  longitudinal  teacher  survey 
data  presented  in  Chapter  2,  present  some  additional  details  on  some  of  the  results  presented  in  the 
chapter,  and  discuss  some  additional  analyses  related  to  those  presented  in  the  chapter.  We  begin 
with  a discussion  of  our  analyses  of  the  content  taught,  and  then  turn  to  pedagogy. 


Content  Coverage  and  High  Standards 


As  reported  in  Chapter  2,  we  drew  on  the  baseline  wave  of  the  longitudinal  survey  to 
describe  the  content  teachers  emphasized  in  their  mathematics  and  science  teaching.  We  then 
compared  the  content  emphasized  by  the  teachers  in  our  sample  to  the  content  emphasized  in  the 
National  Assessment  of  Educational  Progress.  In  the  following  sections,  we  describe  the  methods  we 
used  to  carry  out  these  analyses  and  we  present  some  results  to  supplement  those  discussed  in  the 
chapter. 

Measuring  the  Content  Taught 

Our  main  data  on  content  come  from  the  baseline  wave  of  the  longitudinal  survey  of  teacher 
change.  We  characterize  the  content  taught  in  terms  of  two  major  dimensions:  the  topics  covered  and 
the  performance  goals  teachers  hold  for  students. 

In  the  content  section  of  the  survey,  we  asked  teachers  to  describe  the  content  they  taught  in 
the  class  they  chose  to  describe,  using  a two-dimensional  matrix.  Different  forms  of  the  matrix  were 
used  for  elementary,  middle,  and  high  school  mathematics  and  science. 

The  rows  of  the  matrix  contain  a comprehensive  list  of  the  topics  and  subtopics  teachers 
might  cover.  The  columns  of  the  matrix  contain  performance  goals  for  students.  Performance  goals 
are  teachers’  expectations  for  what  students  should  be  able  to  do.  There  are  six  performance  goals  in 
the  matrix:  1)  memorize;  2)  understand  concepts;  3)  perform  procedures;  4)  generate  hypotheses;  5) 
collect,  analyze,  and  interpret  data;  and  6)  make  connections.  A content  area  can  be  defined  as  the 
intersection  of  the  two  dimensions,  topics  and  performance  goals. 

Each  teacher  was  asked  to  follow  several  steps  in  describing  the  teacher’s  course  using  the 
matrix.  First,  the  teacher  indicated  the  amount  of  time  given  to  each  subtopic,  using  a scale  from  0 
(no  time)  through  3 (more  than  two  lessons  or  class  periods).  Then,  the  teacher  indicated  the  relative 
amount  of  emphasis  given  to  each  performance  goal,  when  teaching  the  subtopic,  using  a scale  from 
0 (no  emphasis)  to  3 (sustained  emphasis).  We  used  the  full  matrix  of  data  provided  by  each  teacher 
to  calculate  the  percent  of  the  teacher’s  total  class  time  devoted  to  each  topic  and  subtopic,  each 
performance  goal,  and  each  content  area  (intersection  of  a subtopic  and  performance  goal). 
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To  calculate  the  percent  of  a teacher's  class  time  developed  to  each  content  area,  we  carried 
out  the  following  steps: 

Step  1.  We  calculated  the  number  of  class  periods  that  the  teacher  spent  on  each 
subtopic.  Teachers  indicated  whether  they  gave  each  subtopic  a coverage  of  0 (no  time),  1 (less  than 
one  class/lesson),  2 (one  to  two  classes/lessons),  or  3 (more  than  two  classes/lessons).  To  determine 
the  overall  percent  of  total  instructional  time  over  the  school  year  given  to  each  subtopic,  we  recoded 
the  teachers’  responses  to  reflect  the  number  of  class  periods/lessons  spent  on  each  subtopic.  We 
recoded  a coverage  of  1 as  one-half  class  period/lesson  and  a coverage  of  2 as  one  and  one-half  class 
periods/lessons.  Recoding  the  coverage  for  each  subtopic  in  which  the  teacher  reported  a coverage 
of  3 (more  than  two  classes/lessons)  was  more  difficult  because  we  do  not  know  for  sure  how  many 
more  than  two  classes/lessons  might  have  been  devoted  to  the  subtopic.  To  recode  the  3’s,  we 
assumed  that  each  teacher’s  total  coverage  across  all  subtopics  should  sum  to  180  classes,  the  typical 
number  of  classes  in  a school  year  in  the  study  schools.  We  summed  the  number  of  class  periods 
accounted  for  by  all  subtopics  given  a coverage  of  1 or  2,  and  subtracted  this  from  180  to  determined 
the  number  of  class  periods  remaining.  We  then  divided  this  by  the  number  of  subtopics  given  a 
coverage  of  3,  to  estimate  the  number  of  class  periods  devoted  to  each  class  period  given  a coverage 
of  3. 


Step  2.  We  computed  the  proportion  of  the  year  spent  on  each  subtopic.  We  carried  out 
this  step  by  dividing  the  number  of  class  periods  spent  on  each  subtopic  by  180,  the  total  number  of 
class  periods  in  the  year.  We  then  summed  across  the  subtopics  under  each  topic  to  estimate  the 
proportion  of  time  spent  on  each  topic.  The  results  of  these  analyses  are  presented  in  Exhibit  2.3  for 
middle  school  mathematics,  and  in  Exhibits  2.4  and  2.5  for  the  subtopics  related  to  measurement  for 
grades  K-12. 

Step  3.  We  computed  the  distribution  of  instructional  time  across  performance  goals 
for  each  subtopic.  For  each  subtopic  the  teacher  covered,  the  teacher  was  asked  to  report  the 
relative  emphasis  given  to  each  of  the  six  performance  goals,  using  a scale  from  0 (no  emphasis)  to  3 
(sustained  emphasis).  For  each  teacher  and  each  subtopic,  we  determined  the  relative  emphasis 
given  to  each  performance  goal  by  dividing  the  emphasis  given  to  that  performance  goal  by  the  sum 
of  the  emphases  given  to  all  six  performance  goals.  For  example,  if  a teacher  indicated  that  she 
taught  addition,  and  she  reported  an  emphasis  of  2 for  memorizing,  an  emphasis  of  3 for 
understanding  concepts,  an  emphasis  of  1 for  making  connections,  and  an  emphasis  of  0 for  the  other 
performance  goals,  the  total  emphasis  for  the  performance  goals  for  addition  would  be  2+3+ 1=6. 

The  relative  emphasis  on  memorizing  would  be  2/6,  or  0.33.  This  relative  emphases  on  performance 
goals,  aggregated  across  teachers  and  subtopics,  is  presented  in  Exhibit  2.6  for  elementary  school 
mathematics. 

Step  4.  We  calculated  the  relative  emphasis  given  to  each  content  area  (subtopic  by 
performance  goal).  We  carried  this  out  for  each  content  area  (subtopic  by  performance  goal)  by 
multiplying  the  proportion  of  total  instructional  time  spent  on  the  subtopic  by  the  relative  emphasis 
given  to  the  performance  goal  within  the  subtopic.  The  total  number  of  content  areas  given  at  least 
some  coverage  is  summarized  in  Exhibit  2.8  for  elementary,  middle,  and  high  school  mathematics 
and  science,  and  the  relative  emphasis  given  to  specific  content  areas  is  presented  in  Exhibit  2.9b  for 
elementary  school  science. 

Measuring  the  Content  Emphasized  in  the  NAEP 
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To  determine  the  consistency  of  the  content  taught  with  high  standards,  we  compared  the 
percent  of  time  each  teacher  devoted  to  specific  topics,  performance  goals,  and  content  areas  to  the 
relative  emphasis  given  to  the  same  topics,  performance  goals,  and  content  areas  by  the  National 
Assessment  of  Educational  Progress  (NAEP).  Mathematics  and  science  generally  are  tested  in  every 
other  NAEP  administration,  or  every  four  years.  The  data  used  for  these  analyses  were  the  1996 
mathematics  and  science  NAEP  tests. 

To  determine  the  relative  amount  of  emphasis  given  by  the  NAEP  to  each  subtopic, 
performance  goal,  and  content  area  in  our  elementary,  middle,  and  high  school  science  and 
mathematics  matrices,  we  reviewed  the  full  set  of  NAEP  mathematics  and  science  items  for  the  1996 
tests  for  fourth,  eighth,  and  twelfth  grade  (see  Exhibit  D.  1). 

EXHIBIT  D.1 

Number  of  NAEP  Items  Rated 


Mathematics 

Science 

Fourth  Grade 

185 

140 

Eighth  Grade 

241 

192 

Twelfth  Grade 

248 

206 

Two  science  curriculum  experts  reviewed  the  full  set  of  NAEP  science  items  to  determine  the 
subtopics  and  performance  goals  each  item  tapped,  using  the  matrix  of  subtopics  and  performance 
goals  in  the  longitudinal  surveys.  Each  expert  was  asked  to  identify  from  one  to  three  content  areas 
(i.e.,  subtopic  by  performance  goal  cells)  that  each  item  tapped.  For  example,  an  expert  might 
indicate  that  a particular  4lh'grade  NAEP  item  tapped  memorization  of  a moon  fact — a single  content 
area.  Or,  the  expert  might  indicate  that  the  item  tapped  three  performance  goals  pertaining  to  the 
moon — memorization,  understanding,  and  performing  procedures.  Or,  the  expert  might  indicate  that 
the  item  tapped  three  different  subtopics,  each  with  an  emphasis  on  performing  procedures.  Two 
mathematics  curriculum  experts  conducted  similar  analyses  for  the  NAEP  mathematics  items. 

The  experts  rated  the  items  on  the  4th'grade  NAEP  using  the  matrix  of  subtopics  and 
performance  goals  in  the  elementary  school  survey;  they  rated  the  items  on  the  8th'grade  NAEP  using 
the  matrix  in  the  middle  school  survey;  and  they  rated  the  items  on  the  12th'grade  NAEP  using  the 
matrix  in  the  high  school  survey. 

We  examined  the  inter-rater  reliability  of  the  ratings  provided  by  the  two  mathematics 
experts,  as  well  as  the  reliability  of  the  ratings  by  the  two  science  experts.  Exhibit  D.2  displays  the 
percent  of  the  ratings  for  which  the  two  mathematics  or  science  experts  agreed  exactly  in  their 
judgment  of  the  performance  goals  or  subtopics  each  item  tapped.  The  results  indicate  that  the  raters 
were  reasonably  similar  in  their  rating  of  the  performance  goals  emphasized  by  the  NAEP  items, 
with  the  percent  of  agreement  among  ratings  ranging  from  nearly  45  percent  to  nearly  65  percent, 
depending  on  subject  (mathematics  or  science)  and  grade  level  (4lh‘grade,  8th‘grade,  or  12th  grade). 
The  degree  of  agreement  among  ratings  for  subtopics  is  somewhat  lower,  but  still  substantial,  given 
the  large  number  of  subtopics  involved.  For  example,  the  content  matrix  for  the  12lh  grade  science 
survey  contains  191  subtopics,  some  of  which  are  quite  closely  related  (e.g.,  cell  structure/function 
and  cell  metabolism),  and  thus  we  would  not  anticipate  perfect  agreement  among  ratings. 
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EXHIBIT  D.2 


Inter-rater  Reliability  for  Ratings  of  NAEP  Items 
(Percent  Agreement  between  Ratings) 


Mathematics 

Science 

Fourth 

Eighth 

Twelfth 

• Fourth 

Eighth 

Twelfth 

Grade 

Grade 

Grade 

Grade 

Grade 

Grade 

Subtopics 

51.70 

50.18 

45.85 

39.37 

44.17 

44.12 

Performance  Goals 

44.49 

51.97 

62.82 

64.57 

54.17 

64.71 

To  determine  the  relative  amount  of  emphasis  given  by  the  full  set  of  NAEP  items  to  each 
content  area  (i.e.,  each  performance  goal  for  each  subtopic),  we  took  the  following  steps. 

Step  1.  We  assigned  weights  to  the  ratings  given  to  each  item  by  each  rater.  To  make 
use  of  the  ratings  provided  by  the  experts,  we  needed  to  convert  the  ratings  they  provided  into  an 
estimate  of  the  proportion  of  emphasis  given  by  NAEP  items  to  each  content  cell.  The  process  of 
converting  the  ratings  to  a proportion  of  emphasis  is  complicated  by  several  factors.  First,  NAEP 
items  vary  in  the  expected  time  required  to  complete  them.  In  estimating  the  relative  emphasis  given 
by  the  NAEP  to  each  content  cell,  we  weighted  items  in  proportion  to  NAEP’s  estimate  of  the 
completion  time.  Second,  although  in  most  cases  two  experts  rated  each  item,  in  a few  cases  one  of 
our  experts  skipped  an  item.  Third,  items  vary  in  the  number  of  ratings  each  of  our  experts  assigned. 
For  some  items,  raters  assigned  only  a single  content  cell,  while  for  others,  raters  assigned  as  many 
as  three  cells. 

To  take  these  factors  into  account,  we  assigned  a weight  to  each  rating  by  each  rater  for  each 
item  rated.  The  weight  is  composed  of  three  factors:  1)  the  relative  time  students  were  expected  to 
spend  on  the  item  according  to  NAEP,  with  a weight  of  one  indicating  an  item  with  one  correct 
answer,  such  as  a multiple-choice  item  and  a weight  of  2 to  5 indicating  a constructed-response  item 
with  partial  credit  possible1;  2)  the  number  of  raters  evaluating  the  item;  and  3)  the  number  of  ratings 
each  rater  gave  the  item.  The  system  of  weights  ensures  that  more  challenging  items  are  given  more 
weight,  that  items  get  equal  weight  whether  they  were  rated  by  one  rater  or  two,  and  that  each  rater 
has  equal  weight,  regardless  of  whether  the  rater  gave  the  item  one,  two,  or  three  ratings. 

Exhibit  D.3  illustrates  the  weights  assigned  for  the  ratings  given  to  a single  item  by  our  two 
raters.  In  the  example,  the  item  is  multiple  choice  in  form  with  only  one  correct  response;  it  was 
rated  by  both  raters.  Rater  1 assigned  the  item  to  two  content  cells  and  rater  2 assigned  the  item  to 
three  cells.  The  item  is  multiple  choice,  so  the  weight  reflecting  the  expected  completion  time=l. 


We  used  the  number  of  Item  Response  Theory  (IRT)  parameters  as  the  weight  for  the  items.  More  complex  items 
require  more  IRT  parameters,  so  they  are  weighed  more  heavily.  Dichotomous  items  (i.e.,  items  that  are  graded  as 
either  right  or  wrong,  including  all  multiple-choice  and  some  short-answer  items)  have  one  IRT  parameter,  and  so 
are  weighted  by  one.  Short  constructed-response  items  are  questions  that  “required  students  to  provide  answers  to 
computation  problems  or  to  describe  solutions  in  one  or  two  sentences”;  these  items  are  expected  to  take  two  to 
three  minutes  to  complete  (Reese  et  al.,  1997,  p.  79).  Extended  constructed-response  items  “required  students  to 
provide  longer  answers  (e.g.,  a description  of  possibilities,  a more  involved  computational  analysis,  or  a description 
of  a patterns  and  its  implications)”;  these  items  are  expected  to  take  five  minutes  to  complete  (Reese  et  al.,  1997, 

P-  79). 
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Both  raters  rated  the  item,  so  the  number  of  raters=2,  and  the  weight  for  each  rater  is  1/2,  or  0.5.  The 
first  rater  assigned  two  ratings  to  the  item;  thus,  for  the  first  rater,  the  weight  for  each  rating=l/2,  or 
0.5.  The  second  rater  assigned  three  ratings  to  the  item;  so,  for  the  second  rater,  the  weight  for  each 
rating  is  1/3  or  0.33. 

The  three  weights  pertaining  to  each  rating  of  each  item  by  each  rater  are  multiplied  to  obtain 
a total  weight  for  the  rating.  Thus,  in  the  example  in  Exhibit  D.3,  the  weight  assigned  to  the  first 
rating  given  by  the  first  rater  = 0.5*0.5=0.25. 

EXHIBIT  D.3 

Example  of  Weights  for  Ratings  of  One  Item 


NAEP 

Item  weight 

Rater  Weight 

Number  of 
Ratings  Weight 

Total  Weight 

Rater  1 

Rating  1 

1 

.50 

.50 

.25 

Rating  2 

1 

.50 

.50 

.25 

Rater  2 

Rating  1 

1 

.50 

.33 

.165 

Rating  2 

1 

.50 

.33 

.165 

Rating  3 

1 

.50 

.33 

.165 

Step  2.  We  determined  the  relative  emphasis  given  by  the  NAEP  to  each  subtopic.  To 

carry  out  this  step,  we  summed  the  total  number  of  weighted  rating  points  given  to  each  subtopic  and 
divided  by  the  total  number  of  rating  points  given  to  all  subtopics.  The  relative  emphasis  given  by 
the  NAEP  to  topics  is  presented  in  Exhibit  2.3  for  middle  school  mathematics. 

Step  3.  We  determined  the  relative  emphasis  given  by  the  NAEP  to  each  performance 
goal.  To  carry  out  this  step,  we  summed  the  total  number  of  weighted  rating  points  given  to  each 
performance  goal  and  divided  by  the  total  number  of  rating  points  given  to  all  performance  goals. 
The  relative  emphasis  given  by  the  NAEP  to  each  performance  goal  is  presented  in  Exhibit  2.6  for 
elementary  school  mathematics. 

Step  4.  We  determined  the  relative  emphasis  given  by  the  NAEP  to  each  content  cell 
(subtopic  by  performance  goal).  To  carry  out  this  step,  we  summed  the  total  number  of  rating 
points  given  to  each  content  cell,  and  divided  by  the  total  rating  points  given  to  all  cells.  This 
relative  emphasis  given  by  the  NAEP  to  content  areas  is  summarized  in  Exhibit  2.8  and  presented  in 
more  detail  for  elementary  science  in  Exhibit  2.9a. 

Step  5.  We  determined  the  alignment  of  teachers’  reports  and  the  NAEP.  As  a final 
step  in  our  analysis,  we  computed  an  index  of  alignment  for  each  teacher,  measuring  the  degree  to 
which  the  teacher’s  relative  emphasis  on  each  content  cell  matches  the  NAEP.  We  based  the  index 
of  alignment  on  Gamoran,  Porter,  Smithson,  and  White  (1997).  For  each  teacher,  the  index  is 
computed  as  the  sum,  across  content  areas,  of  the  absolute  value  of  the  difference  between  the 
emphasis  given  by  the  teacher  to  the  content  area  and  the  emphasis  given  by  the  NAEP.  The 
absolute  value  is  required  because  the  index  is  designed  to  capture  cells  for  which  the  teachers  give 
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more  emphasis  than  NAEP,  as  well  as  those  for  which  they  give  less  emphasis.  The  index  is  scaled  to 
range  from  0.0  to  1.0.  The  results  of  the  alignment  index  are  presented  in  Exhibit  2.10. 

Supplementary  Tables 

Topic  emphases  and  high  standards.  In  Exhibit  2.3,  we  described  teachers'  emphases  on 
topics,  using  middle  school  mathematics  teachers  as  an  example.  Here,  we  present  the  topics 
emphasized  by  teachers  in  the  remaining  five  subgroups:  elementary  and  high  school  mathematics; 
and  elementary,  middle,  and  high  school  science. 

EXHIBIT  D.4a 

Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey 
on  Emphasis  on  Topics  - Elementary  School  Math  (n=74) 
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Percent  of  Emphasis  on  Topics 


EXHIBIT  D.4b 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 

on  Topics  - High  School  Math  (n=69) 
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EXHIBIT  D.4c 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey 
on  Emphasis  on  Topics  - Elementary  School  Science  (n=69) 


Topics 


392 
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EXHIBIT  D.4d 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 

on  Topics  - Middle  School  Science  (n=41) 


■ NAEP 
□ Teachers 
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EXHIBIT  D.4d  (Continued) 

Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 

on  Topics  - Middle  School  Science  (n=41) 


Topics 


394 


EXHIBIT  D.4e 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 

on  Topics  - High  School  Science  (n=64) 
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EXHIBIT  D.4e  (Continued) 

Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 

on  Topics  - High  School  Science  (n=64) 


Performance  goals  and  high  standards.  In  Exhibit  2.6,  we  described  the  performance 
goals  teachers  hold  for  students,  using  elementary  school  mathematics  teachers  as  an  example.  Here 
we  present  the  performance  goals  of  teachers  for  the  remaining  five  subgroups:  middle  and  high 
school  mathematics;  and  elementary,  middle,  and  high  school  science. 
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EXHIBIT  D.5a 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 
on  Performance  Goals  - Middle  School  Math  (n=38) 


■ NAEP 
□ Teacher 
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Percent  of  Emphasis  on  Performance  Goals 


EXHIBIT  D.5b 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 
on  Performance  Goals  - High  School  Math  (n=69) 
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EXHIBIT  D.5c 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Relative 
Emphasis  on  Performance  Goals  - Elementary  School  Science  (n=69) 


100%  - 

90%  - 
80%  - 
70%  - 
60%  - 
50%  - 


■ NAEP 
□ Teacher 


40% 


10%  - 


Performance  Goals 


399 


EXHIBIT  D.5d 


Comparison  of  NAEP  and  Teachers  on  the  Longitudinal  Teacher  Survey  on  Emphasis 
on  Performance  Goals  - Middle  School  Science  (n=41) 
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EXHIBIT  D.5e 


Comparison  of  NAEP  and  Teachers  in  the  Longitudinal  Teacher  Survey  on  Emphasis 
on  Performance  Goals  - High  School  Science  (n=64) 


■ NAEP 
□ Teacher 


Performance  Goals 


Exhibit  2.7  in  Chapter  2 displays  significant  differences  in  performance  goals  among  schools 
and  types  of  teachers.  We  conducted  MANOVAs  to  determine  whether  school  level  (i.e., 
elementary,  middle,  high  school),  subject  (i.e.,  mathematics  or  science),  or  poverty  level  of  the 
school  affects  th.,  ,_ypes  of  performance  expectations  teachers  held  for  students.  The  results  are 
shown  in  Exhibit  D.6,  below. 
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Effects  of  School  Level,  Subject,  and  Poverty  on  Teachers’  Emphasis  on  Performance  Goals 

F-Values,  Tukey  Pairwise  Contrasts,  and  df  (n=355) 
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Content  emphasis  and  high  standards.  Chapter  2,  Exhibit  2.9,  shows  the  relative 
emphasis  teachers  and  the  NAEP  place  on  content  areas  for  elementary  school  science.  Here,  we 
display  the  same  information  for  the  remaining  five  subgroups:  elementary,  middle,  and  high  school 
mathematics;  and  middle  and  high  school  science. 

EXHIBIT  D.7a 

Emphasis  on  Content  Areas  in  Fourth-Grade  Math  NAEP  Items 
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EXHIBIT  D.7b 


Emphasis  on  Content  Areas,  Reported  by  Elementary  School  Math  Teachers  in  the 

Longitudinal  Teacher  Survey  (n=74) 


TMEAN 


under  .01 


.01  to  .02 


.02  to  .03 


m 


over  .03 
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EXHIBIT  D.7c 


Emphasis  on  Content  Areas  in  Eighth-Grade  Math  NAEP  Items 


Memorize 


Number  Sense 


Computation 


Measurement 


Analysis/Prob 


Pre-Algebra 


Basic  Algebra 


Advanced  Algebra 


Geometry 


Technology 


NMEAN  1 ] under  .01  1 3.01  to  0?_ 
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over .03 
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EXHIBIT  D.7d 


Emphasis  on  Content  Areas,  Reported  by  Middle  School  Math  Teachers  in  the 

Longitudinal  Teacher  Survey  (n=38) 


TMEAN  | | under  .01 
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EXHIBIT  D.7e 


Emphasis  on  Content  Areas  in  Twelfth-Grade  Math  NAEP  Items 


NMEAN  1 1 under  .01  [ j .01  to  .02  ||~H  H~t  1 1 1 fi  02  to  .03  [ ''  '1  over  .03 
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EXHIBIT  D.7f 


Emphasis  on  Content  Areas,  Reported  by  High  School  Math  Teachers 
in  the  Longitudinal  Teacher  Survey  (n=69) 


Consumer  Application 


Memorize 


Understand 


Perform 

procedures 


Generate 

hypotheses 


Collect 

data 


Make 

connections 


Data  Analysis 


Pre-Algebra 


Basic  Algebra 


Advanced  Algebra 


Functions 


Geometry 


Advanced  Geometry 


Trigonometry 


Statistics 


Probability 


Finite  Math 


Analysis 


Technology 


TMEAN 


under  .01 


.01  to  .02 


.02  to  .03 


over  .03 
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EXHIBIT  D.7g 

Emphasis  on  Content  Areas  in  Eighth-Grade  Science  NAEP  Items 
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EXHIBIT  D.7h 

Emphasis  on  Content  Areas  in  Middle  School  Science,  Reported  in  the  Longitudinal 

Teacher  Survey (n=41) 
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EXHIBIT  D.7i 

Emphasis  on  Content  Areas  in  Twelfth-Grade  Science  NAEP  Items 


Nature  of  Science 


Memorize 


Understand 


Perform 

procedures 


Generate 

hypotheses 


Collect  data 


Make 

connections 


Technology 


Measurement 


Living  Systems 


Plants 


Animals 


Humans 


Evolution 


Reproduction 


Ecology 


Energy 


Motion  & Forces 


Electricty 


Waves 


Kinetics  & Equilibri 


Properties  of  Matter 


Earth  Systems 


Astronomy 


Meteorology 


Elements 


Chem  Formulas  & Reac 


Acids,  Bases,  Salts 


Environmental  Chemis 


Nuclear  Chemistry 


TMEAN 


under  .01 


.01  to  .02 


.02  to  .03 
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EXHIBIT  D.7j 


Emphasis  on  Content  Areas,  Reported  by  High  School  Science  Teachers  in  the 

Longitudinal  Teacher  Survey  (n=64) 
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Living  Systems 

Biochemistry 

Plants 

Animals 

Humans 
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Motion  & Forces 

Electricity 
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Kinetics  & Equilib 

Matter 

Earth  Systems 

Astronomy 

Meteorology 
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Chemistry 

Mole 

Acids.  Bases.  Salts 

Org  Chem 

Environmental  Sci 

Nuclear  Chem 
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Alignment  between  content  emphases  and  high  standards.  Exhibit  2. 10  in  Chapter  2 
displays  the  degree  of  alignment  between  teachers'  instructional  emphases  and  NAEP  emphases. 
Here,  Exhibit  D.8  provides  the  means  and  standard  deviations  for  the  graphic  representation  of 
alignment  in  Exhibit  2. 10. 


EXHIBIT  D.8 


Alignment  between  Teachers'  Instruction  and  NAEP  Emphasis  (n=355) 


r 

Mathematics 

Science 

Index  : . 

. Elementary 
School 

Middle 

School 

High  School 

: Elementary 
School 

Middle 

School 

High  School 

Mean 

27.1 

28.8 

19.2 

14.7 

15.0 

11.5 

Standard 

Deviation 

10.8 

7.5 

6.0 

8.2 

5.0 

3.6 

To  better  understand  the  conditions  under  which  teachers'  instruction  aligned  with  NAEP,  we 
examined  the  relationship  between  alignment  and  the  following  factors. 

• School  level:  Is  there  a difference  among  elementary,  middle,  and  high  school 
teachers  in  their  alignment  to  NAEP?  Using  the  general  linear  models  procedure  with 
our  sample  of  354  teachers,  across  three  types  of  schools,  we  found  that  alignment 
differs  by  school  level  (F=  16.55***,  df=2,  351). 

• Subject : Is  there  a difference  between  mathematics  and  science  teachers?  Using  the 
general  linear  models  procedure  with  our  sample  of  354  teachers,  across  two  subjects,  we 
found  that  alignment  differs  by  subject  (F=159.56***,  df=l,  352). 

• Specific  schools:  Is  there  a difference  among  schools  in  how  well  teachers'  instruction  is 
aligned  with  NAEP?  Using  the  general  linear  models  procedure  with  our  sample  of  354 
teachers,  in  29  schools,  we  found  that  schools  differ  significantly  in  how  well  their 
teachers'  instruction  aligns  with  NAEP  (F=2.10**,  df=28,  325). 

• Grade  level:  For  example,  is  there  a difference  among  ninth-,  tenth-,  eleventh-,  and 
twelfth-grade  science  teachers  in  how  well  their  instruction  is  aligned  with  NAEP?  We 
used  the  general  linear  models  procedure  to  examine  this  question  within  each  subgroup 
(e.g.,  high  school  science).  We  found  that,  with  the  exception  of  elementary  school 
mathematics,  alignment  does  not  differ  much  across  grades  within  schools  at  the  same 
level  (i.e.,  elementary,  middle,  or  high;  see  Exhibit  D.9). 

• School  poverty  level:  Is  there  a difference  among  teachers  in  high  poverty  versus  low 
poverty  schools  in  how  well  their  instruction  aligns  with  NAEP?  We  used  the  general 
linear  models  procedure  to  examine  this  question  within  each  subgroup  (e.g.,  high  school 
science).  We  found  that  alignment  does  not  differ  much  between  high-  and  low-poverty 
schools  (see  Exhibit  D.9). 
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EXHIBIT  D.9 


Effects  of  Grade  Level  and  Subject  on  Alignment  between  Teachers'  and  NAEP's 
Content  Emphases  MANOVA  Tests:  F-Values  and  Degrees  of  Freedom  (df)  (n=355) 


Mathematics 

Science 

Elementary?]pJ 
:':'f  School  || 

Middle  ^ 
: School 

High  School  ;: 

IgEleihentary  • 
School 

Middled,'; 

School 

: High  School 

Grade  Level 

24.93*** 

.19 

2.18 

1.91 

.55 

1.12 

df=5, 62 

df=2, 35 

df=3, 65 

df=5, 58 

df=2, 37 

df=4,  57 

School 

.42 

4.07 

.11 

.34 

.87 

.39 

Poverty 

df=1, 72 

df=1,36 

df=1 , 67 

df=1 , 66 

df=1,39 

df=1 , 62 

* Significant  at  p<.05 

**  Significant  at  pc.01 

***  Significant  at  p<.001 


Pedagogy  and  High  Standards 


In  the  sections  below,  we  discuss  the  scales  measuring  pedagogical  approaches  discussed  in 
Chapter  2,  and  we  then  present  several  results  that  supplement  the  material  in  the  chapter. 

Pedagogical  approaches 

We  conducted  analyses  on  a series  of  items  in  the  longitudinal  survey  about  teachers' 
pedagogical  strategies.  (See  Exhibit  D.10  for  the  items  involved.) 
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EXHIBIT  D.10 


Pedagogy  Questions  Drawn  from  Middle  School  Mathematics  Survey 


1 . Indicate  the  percentage  of  time  in  the  target  class  you  spent  on  math  instruction  in  each  of  the  following  activities. 
(Note:  Total  should  sum  to  1 00%.) 


Teacher  Activities 


Percent  of 
Instr.  Time 


a.  Lecturing  to  the  class 

b.  Providing  demonstrations  to  the  class  (including 

lab  demonstrations) 

c.  Leading  whole  class  discussions 

d.  Working  with  students  in  small  groups 

e.  Working  with  students  individually 

f.  Performing  routine  administrative  tasks  (e.g.,  taking 

attendance,  making  announcements,  etc.) 

g.  Helping  students  with  experiments,  projects,  or  

other  hands-on  experiences 

h.  Other:  (please  specify) 

TOTAL  100% 

2.  Indicate  the  percentage  of  class  time  spent  on  math  instruction  that  the  typical  student  is  engaged  in  each  of  the 
following  activities.  (Note:  Total  should  sum  to  100%.) 


Percent  of 

Student  Activities  Instr.  Time 

a.  Listening/taking  notes/observing  demonstrations 

b.  Engaged  in  discussion 

c.  Doing  lab  or  field  work 

d.  Completing  exercises/taking  a test  or  quiz! 

e.  Reading 

f.  Completing  a performance  task,  writing 

g.  Presenting  material  to  the  class 

h.  Other:  (please  specify) 

TOTAL  100% 
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EXHIBIT  D.10  (Continued) 

Pedagogy  Questions  Drawn  from  Middle  School  Mathematics  Survey 


3.  How  often  did  you  have  students  (during  math):  (Circle  one  for  each  line.) 


Almost 

Some 

Most 

Every 

Never 

Lessons 

Lessons 

Lesson 

a.  Work  on  or  review  homework  in  class 

0 

1 

2 

3 

b.  Work  on  paper-and-pencil  exercises  related  to 

the  topic 

0 

1 

2 

3 

c.  Work  on  independent,  long-term  (at  least  one- 

week)  projects 

0 

1 

2 

3 

d.  Work  on  problems  for  which  there  is  no 

immediately  obvious  method  or  solution 

0 

1 

2 

3 

e.  Develop  technical  or  mathematical  writing 
skills,  including  using  equations,  graphs, 
tables,  and  text  together 

0 

1 

2 

3 

f.  Work  on  interdisciplinary  lessons  (e.g., 

writing  journals  in  class) 

0 

1 

2 

3 

g.  Recite  or  drill  orally 

0 

1 

2 

3 

h.  Debate  ideas  or  otherwise  explain  their 

reasoning 

0 

1 

2 

3 

i.  Complete  a short  test  or  quiz  to  review 

previous  lesson 

0 

1 

2 

3 

j.  Use  concrete  models  or  manipulatives 

0 

1 

2 

3 

About  how  often  did  students  use  the  following  as  part  of  math  instruction: 
(Circle  one  for  each  line.) 

Almost 

Some 

Most 

Every 

Never 

Lessons 

Lessons 

Lesson 

a.  Standard  calculators  to  solve  basic  exercises  or 
problems 

0 

1 

2 

3 

b.  Programmable  calculators  to  solve  advanced 

exercises  or  problems 

0 

1 

2 

3 

c.  Graphing  calculators  to  graph  equations  or 

data 

0 

1 

2 

3 

d.  Calculators  or  computers  to  develop  models  or 
simulations 

0 

1 

2 

3 

e.  Calculators  or  computers  for  data  collection 

and  analysis 

0 

1 

2 

3 

f.  Computers  for  drill  and  practice  on  skill 

acquisition 

g.  Computers  to  write  reports 

0 

1 

2 

3 

h.  Computers  to  access  the  internet 

0 

1 

2 

3 

O 

ERLC 


D-32 


41 7 


EXHIBIT  D.10  (Continued) 

Pedagogy  Questions  Drawn  from  Middle  School  Mathematics  Survey 


6.  About  how  often  did  you  interact  with  students  in  the  targeted  class  in  the  following  ways:  (Circle  one  for  each 
line.) 


Almost 

Some 

Most 

Every 

Never 

Lessons 

Lessons 

Lesson 

a. 

Students  work  individually  without  your  ongoing 
assistance 

0 

1 

2 

3 

b. 

Students  work  individually  with  your  ongoing  assistance 

0 

1 

2 

3 

c. 

Work  together  as  a class  with  students 
responding  to  one  another 

0 

1 

2 

3 

d. 

Work  in  pairs  or  small  groups  without  your 
ongoing  assistance 

0 

1 

2 

3 

e. 

Work  in  pairs  or  small  groups  with  your 

ongoing  assistance : 

0 

1 

2 

3 

7.  How  important  were  the  following  assessment  strategies  in  determining  students’  grades  in  this  math  class: 
(Circle  one  for  each  line.) 


Not 

Used 


a.  Objective  tests  (e.g.,  multiple  choice) 0 

b.  Essay  tests 0 

c.  Performance  tasks  or  events 0 

d.  Systematic  observation  of 

students 0 

e.  Math  reports 0 

f.  Math  projects 0 

g.  Homework  assignments 0 

h.  Portfolios 0 


Minor 

Moderate 

Very 

ImDortance 

ImDortance 

ImDortant 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 
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To  identify  patterns  in  teachers'  pedagogical  activities,  we  conducted  factor  analyses  on  the 
full  set  of  items  identified  in  Exhibit  D.10.  The  following  four  factors,  consistent  with  research  on 
pedagogy,  emerged. 

Didactic  instruction,  (alpha  reliability=.  75). 

• Students  working  on  interdisciplinary  lessons  (reverse  coded) 

• Students  using  concrete  models  or  manipulatives  (reverse  coded) 

• Teacher  lecturing  to  class 

• Teacher  working  with  students  in  small  groups  (reverse  coded) 

• Students  listening/taking  notes/observing  demonstration 

• Students  reading  (reverse  coded) 

• Students  completing  a performance  task,  writing  (reverse  coded) 

• Students  presenting  material  to  the  class  (reverse  coded) 

Individual  Seatwork.  (alpha  reliability=.69). 

• Students  working  on  or  reviewing  homework  in  class 

• Students  working  on  paper- and-pencil  exercises  related  to  the  topic 

• Students  reciting  or  drilling  orally 

• Students  completing  a short  test  or  quiz  to  review  previous  lesson 

• Students  work  individually  without  your  ongoing  assistance 

• Students  work  individually  with  your  ongoing  assistance 

• Students  work  in  pairs  or  small  groups  without  your  ongoing  assistance 
Active,  project-centered  instruction,  (alpha  reliability=.67). 

• Students  working  on  independent,  long-term  projects 

• Students  working  on  problems  for  which  there  is  no  immediately  obvious  method  or 
solution 

• Students  developing  technical  or  mathematical/scientific  writing  skills 

• Teacher  working  with  students  individually  (reverse  coded) 
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• Teacher  helping  students  with  experiments,  projects,  or  other  hands-on  experiences 

• Students  doing  lab  or  field  work 

• Students  completing  exercises/taking  a test  or  quiz  (reverse  coded) 

• Students  doing  other  (reverse  coded) 

Discussion-oriented  instruction,  (alpha  reliability=.67). 

• Teacher  leading  whole  class  discussion 

• Students  engaging  in  discussion 

Exhibit  2.13  in  Chapter  2 displays  significant  differences  in  pedagogical  approaches  among 
schools  and  types  of  teachers.  Exhibit  D.l  1 provides  additional  information  concerning  the  ANOVA 
results. 

EXHIBIT  D.11 

Effects  of  School  Level  and  Subject  on  Teachers’  Pedagogical  Approaches 
F-Values,  Tukey  Pairwise  Contrasts,  and  df  (n=355) 


Traditional  I Nontraditional 


Didactic  Individual  Active  Discussion-oriented* 

Seatwork  


School  Level 

42.89***  (df=2, 352) 

5.61** 

(df=2, 352) 

0.55 

(df=2, 352) 

4.23* 

(df=2,  339) 

HS-MS 

2.86* 

(df=1, 210) 

1.92 

(df=1, 210) 

-0.25 

(df=1, 210) 

-1.86 

(df=1 , 202) 

HS-ES 

6.20* 

(df=1, 274) 

2.24* 

(df=1 , 274) 

-0.73 

(df=1 , 274) 

-3.07* 

(df=1 , 265) 

MS-ES 

3.34* 

(df=1, 220) 

0.32 

(df=1 , 220) 

-0.48 

(df=1, 220) 

-1.22 

(df=1, 211) 

Subject 

1.09 

(df=1 , 353) 

31.67***(df=1,  353) 

50.69***  (df=1, 353) 

0.37 

(df=1, 340) 

School  Poverty  Level 

16.65***  (df=1, 352) 

0.00 

(df=1,  352) 

0.01 

(df=1, 352) 

1.06 

(df=1 , 339) 

+ n=342 


* Significant  at  p<.05 
**  Significant  at  pc.01 
***  Significant  at  pc.001 


Supplementary  Tables 

Two  of  the  pedagogy  questions  ask  teachers  to  report  the  percentage  of  time  they  spend  in  a 
selected  class  on  various  activities.  The  results  are  reported  in  Exhibit  D.12. 

EXHIBIT  D.12 

Time  Spent  on  Class  Activities  (n=339) 


Percent  of  Time  Teacher  Spends  on  Instructional  Activities 


Mean 

Standard 

Deviation 

Lecturing 

19.50 

14.49 

Providing  demonstrations 

14.75 

9.40 

Leading  whole-class  discussion 

15.02 

10.95 

Working  with  students  in  small  groups 

15.90 

10.60 

Working  with  students  individually 

12.11 

9.34 

Performing  routine  administrative  tasks 

4.67 

4.71 

Helping  students  with  hands-on  experiences* 

17.29 

13.84 

Other* 

0.71 

2.98 

* n=340 

Percent  of  Time  Students  Spend  on  Learning  Activities 


Mean 

Standard 

Deviation 

Listening/taking  notes/observing* 

25.91 

16.25 

Engaged  in  discussion 

16.07 

9.68 

Doing  lab  or  field  work 

16.30 

14.84 

Completing  exercises/taking  a test  or  quiz 

15.17 

11.62 

Reading 

6.90 

6.54 

Completing  a performance  task,  writing* 

12.22 

9.19 

Presenting  material* 

6.56 

6.55 

Other 

0.71 

3.49 

+ n=338 


421 

o 

ERIC 


D-36 


APPENDIX  E 


SUPPLEMENT  FOR  CHAPTER  3 


This  appendix  provides  supplementary  information  for  the  analyses  reported  in  Chapter  3. 
Each  Exhibit  in  the  appendix  corresponds  to  an  Exhibit  of  the  same  number  in  Chapter  3.  The  boxes 
at  the  bottom  of  each  Exhibit  refer  to  the  Teacher  Activity  Survey  item  numbers  or  composite 
variables  on  which  the  exhibit  is  based.  Teacher  Activity  Survey  item  numbers  begin  with  the  letter 
“A,”  followed  by  the  section  of  the  survey  and  the  item  involved.  For  example,  item  number 
“A3_24a”  refers  to  section  3,  item  24a  of  the  survey.  All  items  referred  to  in  the  chapter  appear  at 
the  end  of  this  appendix,  in  Exhibit  3. 2 Id.  Composite  variables,  which  appear  in  all-capital  letters 
(for  example,  ALEARNJT),  are  defined  in  Exhibit  3.21b.  Due  to  missing  data  caused  by  teacher 
nonresponse  to  particular  survey  questions,  the  number  of  teacher  responses  varies  across  survey 
items. 


EXHIBIT  E.3.1 

Percent  of  Teachers  Reporting  Participation  in  Traditional  and  Reform  Types  of 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=775,  SAHE  Grantee  n=244) 


District 

SAHE  Grantee 

In-District  Workshop  or  Institute 

52.06 

22.93 

College  Course 

4.41 

26.09 

Out-of-District  Workshop  or  Institute 

15.12 

25.34 

Out-of-District  Conference 

7.60 

.31 

Collaborative  or  Network 

5.54 

7.96 

Study  Group 

1.04 

2.92 

Mentoring 

3.70 

7.90 

Committee  or  Task  Force 

8.53 

1.80 

Internship 

.69 

2.58 

Resource  Center 

1.32 

2.16 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998 
Note : “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district 
component  of  the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities 
supported  through  the  SAHE  component  of  the  program. 

Variables : A3_2CAT  (A3_2  with  “other”  responses  categorized  into  standard  response  options) 


E-l 
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EXHIBIT  E.3.2 

Contact  Hours  Provided  by  Eisenhower-assisted  Professional  Development 
Activities,  as  Reported  by  Teachers  (District  n=767,  SAHE  Grantee  n=244) 


Mean 

SD 

n 

F 

df 

P 

Program  Component 

51.28 

1, 1007 

0.00 

District 

25.44 

(35.16) 

767 

SAHE  Grantee 

51.10 

(39.34) 

244 

Type 

2.37 

1, 1007 

0.12 

Traditional 

29.57 

(34.22) 

804 

Reform 

38.51 

(48.75) 

207 

Program  Component  * Type 

5.46 

1, 1007 

0.02 

District ‘Traditional 

22.74 

(28.97) 

609 

District  * Reform 

34.58 

(51.58) 

158 

SAHE  Grantee  * Traditional 

51.73 

(40.23) 

195 

SAHE  Grantee  * Reform 

49.30 

(35.86) 

49 

Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998 
Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Variables : A3JOJ2  =(A3_1G+A3+12) 
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EXHIBIT  E.3.3 


Time  Span  of  Eisenhower-assisted  Activities,  as  Reported  by  Teachers 
(District  n=766,  SAHE  Grantee  n=244) 


Mean 

SD 

n 

F 

df 

P 

Program  Component 

44.36 

1,1007 

0.00 

District 

3.98 

(2.21) 

767 

SAHE  Grantee 

5.61 

(2.09) 

244 

Type 

9.52 

1,1007 

0.00 

Traditional 

4.20 

(2.24) 

804 

Reform 

5.07 

(2.34) 

207 

Program  Component  * Type 

7.08 

1,1007 

0.01 

District  * Traditional 

3.76 

(2.11) 

609 

District  * Reform 

4.89 

(2.36) 

158 

SAHE  Grantee  * T raditional 

5.59 

(2.07) 

195 

SAHE  Grantee  * Reform 

5.67 

(2.21) 

49 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998 
Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Variables:  Time  Span 

EXHIBIT  E.3.4 

Percent  of  Teachers  Reporting  School-level  and  Department  or  Grade-level 
Participation  in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 


Collective  Participation 

District 

SAHE  Grantee 

All  teachers  in  department  or 

19.84 

7.18 

grade  level 

All  teachers  in  school 

18.59 

10.75 

Source.  Mail  Survey  of  Teachers  Participating  in  Eisenhower  supported  Professional  Development  Activities,  1998 
Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component 
of  the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the 
SAHE  component  of  the  program. 

Variables : A3_24c,  A3_24d  


E-3 


EXHIBIT  E.3.5 

Percent  of  Teachers  Reporting  a Major  Emphasis  on  Mathematics  and  Science 
Content  Knowledge  in  Eisenhower-assisted  Professional  Development  Activities 

(District  n=754,  SAHE  Grantee  n=243) 


Mean 

SD 

n 

F 

df 

P 

Program  Component 

21.35 

1,993 

0.00 

District 

1.36 

(0.72) 

754 

SAHE  Grantee 

1.63 

(0.57) 

243 

Type 

1.39 

1,993 

0.24 

Traditional 

1.41 

(0.69) 

794 

Reform 

1.49 

(0.72) 

203 

Program  Component  * Type 

0.01 

1,993 

0.91 

District  * Traditional 

1.34 

(0.72) 

600 

District  * Reform 

1.41 

(0.74) 

154 

SAHE  Grantee  * Traditional 

1.61 

(0.56) 

194 

SAHE  Grantee  * Reform 

1.69 

(0.62) 

49 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Variables:  D2_CO  (A3_13f) 

EXHIBIT  E.3.6 

Percent  of  Teachers  Reporting  That  They  Had  Opportunities  to  Observe  or  Be 
Observed  Teaching  in  Eisenhower-assisted  Professional  Development  Activities 

(District  n=783,  SAHE  Grantee  n=244) 


Types  of  Opportunities  to  Observe  and  Be  Observed  Teaching 

District 

SAHE  Grantee 

Teacher  received  coaching 

9.63 

21.58 

Leader  observed  teacher  teaching 

4.56 

16.49 

Others  observed  teacher  teaching 

10.11 

15.17 

Teacher’s  classroom  observed 

5.30 

21.56 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Variables : A3,6b,  A3_6d,  A3_6e,  A3_35d 
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EXHIBIT  E.3.7 

Percent  of  Teachers  Reporting  Opportunities  to  Plan  Classroom  Implementation  in 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 


Types  of  Opportunities  to  Plan  Classroom  Implementation 

District 

SAHE  Grantee 

Practiced  in  simulated  conditions 

28.95 

38.11 

Held  formal  meetings 

31.97 

50.53 

Communicated  with  leader 

36.35 

53.51 

Held  informal  meetings 

46.70 

52.64 

Developed  lesson  plans 

29.49 

46.43 

Source.  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note.  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component 
of  the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the 
SAHE  component  of  the  program. 

Variables'.  A3_6a,  A3_6c,  A3„6f,  A3_6h,  A3„6i 


EXHIBIT  E.3.8 

Percent  of  Teachers  Reporting  Opportunities  to  Examine  Student  Work  in 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 


Types  of  Opportunities  to  Examine  Student  Work 

District 

SAHE  Grantee 

Teacher  reviewed  student  work 

18.85 

26.40 

Scored  assessments 

9.42 

13.10 

Leader/others  reviewed  student  work 

11.36 

17.84 

Student  outcomes  evaluated 

9.07 

12.19 

Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Variables'.  A3_27m,  A3_27n,  A3_6g,  A3_35e 


o 

ERLC 


E-5 
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EXHIBIT  E.3.9 

Percent  of  Teachers  Reporting  Opportunities  to  Present,  Lead,  and  Write  in 
Eisenhower-assisted  Professional  Development  Activities 
(District  n=783,  SAHE  Grantee  n=244) 


Types  of  Opportunities  to  Present,  Lead,  and  Write 

District 

SAHE  Grantee 

Gave  lecture  or  presentation 

18.27 

42.32 

Conducted  a demonstration 

24.03 

42.58 

Led  a whole-group  discussion 

8.33  . 

9.89 

Led  a small-group  discussion 

16.58 

20.83 

Wrote  a report 

14.75 

43.97 

Source.  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 
opportunity  to  give  a lecture  or  presentation.  Each  bar  and  the  number  on  the  top  of  it  represent  the  percent  of  teachers  for  each 
category. 

Note.  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Variables:  A3_27e,  A3_27f,  A3„27g,  A3_27h,  A3_27j 


EXHIBIT  E.3.10 

Number  of  Opportunities  for  Active  Learning  in  Eisenhower-assisted  Professional 
Development  Activities,  as  Reported  by  Teachers 
(District  n=767,  SAHE  Grantee  n=244) 


Mean 

SD 

n 

F dt 

P 

Program  Component 

38.39  1, 1007 

0.00 

District 

3.60 

(3.49) 

767 

SAHE  Grantee 

5.81 

(3.94) 

244 

Type 

1.45  1,1007 

0.23 

Traditional 

3.96 

(3.64) 

804 

Reform 

4.69 

(3.99) 

207 

Program  Component  * Type 

3.98  1;  1007 

0.05 

District  * Traditional 

3.38 

(3.36) 

609 

District  * Reform 

4.35 

(3.88) 

158 

SAHE  Grantee  * Traditional 

5.87 

(3.89) 

195 

SAHE  Grantee  * Reform 

5.63 

(4.20) 

49 

Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE  component 
of  the  program. 

Scale  Definition:  ALEARN_T=  (A3_6a  + A3_6c  + A3_6f  + A3  6h  + A3  6i)  + 5/4*(A3  6b  + A3  6d  + A3  6e  + A3  35D)  + 
(A3_27e+A3_27f+A3_27g+A3_27h+A3_27j) 

Scale  Reliability  : .84 
Variables:  ALEARN.T 
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EXHIBIT  E.3.1 1 

Percent  of  Teachers  Reporting  That  Eisenhower-assisted  Professional  Development 
Activities  are  Related  to  Their  Other  Professional  Development  Experiences 
(District  n=748  to  760,  SAHE  Grantee  n=238  to  239) 


Relationship  to  Other  Experiences 

District 

SAHE  Grantee 

Consistent  with  goals 

78.99 

87.91 

Build  on  earlier  activities 

35.00 

38.80 

Followed  up  with  additional  activities 

52.88 

69.56 

Source.  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program.  Due  to  missing  data  for  some  items,  the  district  n ranges  from  748  to  760  and  the  SAHE  Grantee  n 
ranges  from  238  to  239. 

Variables'.  A3_34a  A3_34c  A3_34d 


EXHIBIT  E.3.1 2 

Percent  of  Teachers  Reporting  That  Eisenhower-assisted  Activities  are  Aligned  with 
State  and  District  Standards,  Frameworks,  and  Assessments  (District  n=748  to  753, 

SAHE  Grantee  n=238) 


Alignment  with  State  and  District  Standards  and  Assessments 

District 

SAHE  Grantee 

Designed  to  support  state  and  district  standards 

79.69 

79.00 

Designed  to  support  state  and  district  assessments 

68.16 

65.83 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program.  Due  to  missing  data  on  some  items,  the  district  n ranges  from  748  to  753;  the  SAHE  Grantee  n is  238. 
Variables:  A3_34e  A3_34f  . 


BEST  COPY  AVAILABLE 


A2S- 
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EXHIBIT  E.3.13 

Percent  of  Teachers  Reporting  That  They  Communicated  with  Other  Teachers  about 
Their  Experiences  in  the  Eisenhower-assisted  Professional  Development  Activities 

(District  n=783,  SAHE  Grantee  n=244) 


Types  of  Communication 

District 

SAHE  Grantee 

Discussed  with  other  teachers 

73.00 

85.00 

Discussed  with  administration 

62.80 

66.35 

Communicated  with  teachers  in  other  schools 

40.79 

54.31 

Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the 
SAHE  component  of  the  program. 

Variables : A3_31  A3_32  A3_33 


EXHIBIT  E.3.14 

Degree  to  Which  Eisenhower-assisted  Professional  Development  Activities  Foster 
Coherence,  as  Reported  by  Teachers  (District  n=747,  SAHE  Grantee  n=235) 


Mean 

SD 

n 

F 

df 

P 

Program  Component 

10.41 

1,978 

0.00 

District 

5.94 

(1.92) 

747 

SAHE  Grantee 

6.39 

(1.95) 

235 

Type 

15.34 

1,978 

0.00 

Traditional 

5.92 

(1.84) 

783 

Reform 

6.48 

(2.23) 

199 

Program  Component  * Type 

1.77 

1,978 

0.18 

District  * Traditional 

5.84 

(1.83) 

596 

District  * Reform 

6.27 

(2.22) 

151 

SAHE  Grantee  * Traditional 

6.16 

(1.85) 

187 

SAHE.Grantee  * Reform 

7.02 

(215) 

48 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the 
SAHE  component  of  the  program. 

Scale  Definition:  COHERE_T=I  .5*(A3_34e  + A3_34f)+(A3_34a  + A3_34c  + A3_34d)+(A3_3 1 + A3_32  + A3_33) 

Scale  Reliability  : .7 1 
Variables:  COHERE_T 
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EXHIBIT  E.3.15 

Percent  of  Teachers  Reporting  Enhanced  Knowledge  and  Skills  Due  to  Participation 
in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=731  to  750,  SAHE  Grantee  n=233  to  240) 


Areas  of  Knowledge  and  Skills 

District 

SAHE  Grantee 

In-depth  knowledge  of  math/science 

48.20 

68.14 

Curriculum 

55.92 

64.41 

Instructional  methods 

62.79 

79.16 

Approaches  to  assessment 

45.66 

48.29 

Use  of  technology 

24.48 

50.22 

Approaches  to  diversity 

26.04 

35.12 

Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the 
SAHE  Grantees  component  of  the  program.  Due  to  missing  data  on  some  items,  the  district  n for  ranges  from  73 1 to  750;  the 
SAHE  Grantee  n ranges  from  233  to  240. 

Variables : A4_  1 a A4_lb  A4_lc  A4_ld  A4_le  A4_lf 


EXHIBIT  E.3.16 

Extent  to  Which  Participation  in  Eisenhower-assisted  Professional  Development 
Activities  Enhanced  Knowledge  and  Skills,  as  Reported  by  Teachers 
(District  n=750,  SAHE  Grantee  n=240) 


Mean 

SD 

n 

F 

df  p 

Program  Component 

25.92 

1,986  0.00 

District 

3.19 

(0.89) 

750 

SAHE  Grantee 

3.59 

(0.80) 

240 

Type 

17.41 

1,986  0.00 

Traditional 

3.22 

(0.85) 

792 

Reform 

3.54 

(0.97) 

198 

Program  Component  * Type 

0.05 

1,986  0.81 

District  * Traditional 

3.12 

(0.86) 

600 

District  * Reform 

3.45 

(0.95) 

150 

SAHE  Grantee  * Traditional 

3.51 

(0.75) 

192 

SAHE  Grantee  * Reform 

3.81 

(0.96) 

48 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower- assisted  Professional  Development  Activities,  1998. 

Note:  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower- assisted  activities  supported  through  the 
SAHE  component  of  the  program. 

Scale  Definition : EKS_l=(A4_la  + A4_lb  + A4_lc  + A4_ld  + A4_le  + A4_lf)/6 
Scale  Reliability:  .78 
Variables:  EKS  1 

k 

r 


E-9 


4M 
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EXHIBIT  E.3.17 


Percent  of  Teachers  Reporting  Improvement  in  Classroom  Teaching  Practice  Due  to 
Participation  in  Eisenhower-assisted  Professional  Development  Activities 
(District  n=731  to  750,  SAHE  Grantee  n=233  to  244) 


Types  of  Teaching  Practices 

District 

SAHE  Grantee 

Curriculum 

45.86 

54.31 

Cognitive  challenge 

55.00 

65.68 

Instructional  methods 

57.84 

66.94 

Approaches  to  assessment 

44.73 

34.90 

Use  of  technology 

29.07 

45.74 

Approaches  to  diversity 

30.48 

33.76 

Source : Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of 
the  program.  “SAHE  Grantees"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program.  Due  to  missing  data  on  some  items,  the  district  n ranges  from  73 1 to  750;  the  SAHE  Grantee  n ranges 
from  233  to  240. 

Variables : A4_4a  A4_4b  A4_4c  A4_4d  A4_4e  A4_4f 


EXHIBIT  E.3.18 

Degree  of  Improvement  in  Classroom  Teaching  Practice  Due  to  Participation  in 
Eisenhower-assisted  Professional  Development  Activities,  as  Reported  by  Teachers 

(District  n=767,  SAHE  Grantee  n=244) 


Mean 

SD 

n 

F 

df 

P 

Program  Component 

5.87 

1, 1007 

0.02 

District 

1.27 

(0.80) 

767 

SAHE  Grantee 

1.44 

(0.77) 

244 

Type 

11.67 

1, 1007 

0.00 

Traditional 

1.25 

(0.77) 

804 

Reform 

1.49 

(0.88) 

207 

Program  Component  * Type 

0.00 

1, 1007 

0.95 

District ‘Traditional 

1.21 

(0.78) 

609 

District  * Reform 

1.45 

(0.86) 

158 

SAHE  Grantee  * Traditional 

1.38 

(0.72) 

195 

SAHE  Grantee  * Reform 

1.60 

(0.92) 

49 

Source:  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted  Professional  Development  Activities,  1998. 

Note:  “Districts"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the  district  component  of  the 
program.  “SAHE  Grantees"  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  supported  through  the  SAHE 
component  of  the  program. 

Scale  Definition : ICPT=(A4_4a  + A4_4b  + A4_4c  + A4_4d  + A4_4e  + A4_4f) 

Scale  Reliability.  .87 
Variables:  ICTP 

431 
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EXHIBIT  E.3.19 

Percent  of  Teacher  Participations  in  District  Eisenhower-assisted  Activities 
Compared  to  All  Teachers  in  the  Nation,  Overall  and  by  District  Poverty  (n=1197) 


District  Poverty 

Percent  of  all  teachers 
from  high-poverty 
schools 

Percent  of  Title  II 
participations  from  high- 
poverty  schools 

Overall 

21 

23 

Low 

4 

2 

Medium 

14 

17 

High 

45 

51 

Source.  Results  for  all  teachers  are  based  on  data  on  the  full  population  of  schools  from  the  CCD.  Data  on  Title  II 
participations  are  based  on  the  Mail  Survey  of  Teachers  Participating  in  Eisenhower- assisted  Professional 
Development  Activities,  1998,  including  both  respondents  and  non-respondents. 

Variables.  


EXHIBIT  E.3.20 

Percent  of  Teacher  Participations  in  District  and  SAHE  Grantee  Eisenhower-assisted 
Activities  from  High-poverty  Schools  (District  n=1197,  SAHE  Grantee  n=254) 


Percent  of  Teachers  from 
High  Poverty  Schools 

All  Teachers 

21 

District  Participations 

23 

SAH  E-grantee  Participations 

13 

Source.  Results  for  all  teachers  are  based  on  data  on  the  full  population  of  schools  from  the  CCD.  Data  on 
Title  II  participations  are  based  on  the  Mail  Survey  of  Teachers  Participating  in  Eisenhower-assisted 
Professional  Development  Activities.  1998,  including  both  respondents  and  non-respondents. 

Note.  “Districts”  refers  to  teachers  who  participated  in  Eisenhower-assisted  activities  provided  through  the 
district  component  of  the  program.  "SAHE  Grantees”  refers  to  teachers  who  participated  in  Eisenhower- 
assisted  activities  supported  through  the  SAHE  component  of  the  program. 

Variables. 
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EXHIBIT  E.3.21  a (Continued) 

The  Relationships  of  Features  of  Professional  Development  to  Teacher  Outcomes 
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EXHIBIT  E.3.21b 
VARIABLE  DEFINITIONS 
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ATYPP  Activity  Type(LTraditional,  2=Reform) 

Defined  as  follows: 

If  a3_2a=1  or  a3_2b=1=b  or  a3_2c=1  or  a3_2e=1  then  ATYPE=1 
Else  ATYPE=2 
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EXHIBIT  E.3.21C 

Correlations  among  Teacher  Activity  Variables 
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EXHIBIT  E.3.21d 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 


Section  III:  Description  Of  One  Professional  Development  Activity 

The  following  questions  ask  you  to  describe  the  professional  development  activity  identified  on  the  label  on  the  cover  of  the  survey,  which 
you  participated  in  over  the  past  year. 

In  answering  questions  about  the  identified  activity,  please  include  all  components  of  the  activity,  even  if  they  occurred  at  different  times 
during  the  year.  (For  example,  if  the  activity  was  a summer  institute  with  follow-up  during  the  school  year,  include  both  the  summer 
institute  and  the  follow-up  in  your  answers.) 


2.  Which  of  the  following  best  describes  the  activity?  Choose  only  one  response.  If  more  than  one  response  fits  the  activity,  pick  the 
response  that  best  describes  the  aspect  of  the  activity  in  which  you  spent  the  most  time. 


Participation  in  an  in-district  workshop  or  institute a 

Attendance  at  a college  course b 

Attendance  at  an  out-of-district  workshop  or  institute c 

Participation  in  a teacher  collaborative  or  network d 

Attendance  at  an  out-of-district  conference e 

Working  in  an  internship  or  immersion  activity f 

Working  with  a mentor,  coach,  lead  teacher,  or  observer g 

Use  of  a teacher  resource  center h 

Participation  in  a teacher  committee  or  task  force i 

Participation  in  a teacher  study  group j 

Other  (please  specify) k 


5.  As  part  of  the  professional  development  activity,  including  any  preliminary  and  follow-up  sessions,  did  you  have  the  opportunity  to  try 
out  what  you  learned  in  your  classroom  and  obtain  feedback  or  guidance? 


Yes 1 

No 2 (skip  to  question  7) 
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EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 


6.  How  did  this  professional  development  activity  help  you  use  new  skills  in  your  classroom?  (Circle  all  that  apply.) 


Practiced  under  simulated  conditions,  with  feedback a 

Received  coaching  or  mentoring  in  the  classroom b 

Met  formally  with  other  activity  participants  to  discuss  classroom 

implementation c 

My  teaching  was  observed  by  the  activity  leader(s)  and  feedback 

was  provided d 

My  teaching  was  observed  by  other  participants  and  feedback  was 

provided e 

Communicated  with  the  leader(s)  of  the  activity  concerning  classroom 

implementation f 

My  students’  work  was  reviewed  by  participants  or  the 

activity  leader g 

Met  informally  with  other  participants  to  discuss  classroom 

implementation h 

Developed  curricula  or  lesson  plans,  which  other  participants 

or  the  activity  leader  reviewed i 

Other  (specify) j 


7.  Over  what  period  of  time  was  the  activity  spread,  including  the  main  activity  and  any  formal  preliminary  or  follow-up  sessions?  (Circle 
one  response.) 


Less  than  one  day  a 

One  day  b 

Two-four  days  c 

A week  d 

A month  e 

More  than  a month  f 


8.  In  what  month  or  months  did  the  activity  (including  any  preliminary  or  formal  follow-up  sessions)  take  place?  Check  the  appropriate 
month(s)  on  the  timeline  below. 


Before 

June 

July 

Aug 

Sept 

Oct 

Nov 

Dec 

Jan 

Feb 

Mar 

April 

May 

June 

1997 

1997 

1997 

1997 

1997 

1997 

1997 

1997 

1998 

1998 

1998 

1998 

1998 

10.  Between  June  1997  and  the  present,  including  the  main  activity  and  any  preliminary  activities  or  formal  follow-up  sessions,  how  many 
hours  were  you  engaged  in  this  activity  overall? 

hours 
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EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapters 


11.  Is  the  activity  still  continuing? 


Yes 1 

No 2 (skip  to  question  15) 


1 2.  How  many  hours  do  you  expect  to  be  engaged  in  this  activity  between  now  and  the  end  of  the  school  year? 
hours 


1 5.  How  much  emphasis  did  the  activity  give  to  each  of  the  following  areas? 


a.  Curriculum  (e.g.,  units,  texts,  standards) 

b.  Instructional  methods 

c.  Approaches  to  assessment 

d.  Use  of  technology  in  instruction  (e.g.,  computers, 

graphing  calculators) 

e.  Strategies  for  teaching  diverse  student  populations 

(e.g.,  students  with  disabilities,  from  underrepresented 
populations,  economically  disadvantaged,  range  of 
abilities) 

f.  Deepening  your  knowledge  of  mathematics 

g.  Leadership  development 

h.  Other:  (please  specify). 


No 

Minor 

Major 

Emphasis 

Emphasis 

Emphasis 

0 

1 

2 

0 

1 

2 

0 

1 

2 

0 

1 

2 

0 

1 

2 

0 

1 

2 

0 

1 

2 

0 

1 

2 

24.  Which  of  the  following  characterize  the  participants  in  this  activity?  (Circle  all  that  apply.) 


Teachers  as  individuals a 

Teachers  as  representatives  of  their  departments,  grade  level,  or  school b 

All  teachers  in  department  or  grade-level  groupings c 

All  teachers  in  a school  or  set  of  schools d 

Other  configurations  (specify) e 
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EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 


27.  Which  of  the  following  did  you  engage  in  during  the  activity?  (Circle  all  that  apply.) 

Listened  to  a lecture 

Observed  a demonstration  of  a lesson  or  unit 

Participated  in  a whole-group  discussion 

Participated  in  a small-group  discussion 

Gave  a lecture  or  presentation 

Conducted  a demonstration  of  a lesson,  unit,  or  skill 

Led  a whole-group  discussion 

Led  a small-group  discussion 

Engaged  in  extended  problem  solving 

Wrote  a paper,  report  or  plan 

Practiced  using  student  materials 

Developed  or  reviewed  materials 

Reviewed  student  work 

Scored  assessments 

Collaborated  as  a colleague  with  mathematicians 

Used  technology  (computers,  calculators,  or  the  internet) 

Completed  paper-and-pencil  problems  or  exercises 

Assessed  participants’  knowledge  or  skills 

Other:  (please  specify) 


a 

b 

c 

d 

e 

f 

g 

h 

i 

j 

k 

I 

m 

n 

o 

P 

q 

r 

s 


31 .  Have  you  discussed  what  you  learned  with  other  teachers  in  your  school  or  department  who  did 
not  attend  the  activity? 


Yes 1 

No 2 

32.  Have  you  discussed  or  shared  what  you  learned  with  administrators  (e.g.,  principal  or  department  chair)? 

Yes 1 

No 2 


33.  Outside  of  formal  meetings  held  as  part  of  the  activity,  have  you  communicated  with  participants  in  the  activity  who  teach  in  other 
schools? 


Yes 1 

No 2 


E-21 


EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 


34.  To  what  extent  was  the  professional  development  activity: 


a.  Consistent  with  your  own 

goals  for  your  professional 
development.. 

b.  Consistent  with  your  school’s 

or  department’s  plan  to 
change  practice 

c.  Based  explicitly  on  what  you  had 

learned  in  earlier  professional 
development  experiences 

d.  Followed  up  with  activities  that 

built  upon  what  you  learned  in 
this  professional  development 
activity 

e.  Designed  to  support  state  or 

district  standards/curriculum 
frameworks 

f.  Designed  to  support  state  or 

district  assessment 


Not  at  all 


Great  Not 

Extent  Applicable 


na 


na 


na 


na 


3 4 

3 4 


na 

na 


35.  How  was  the  activity  evaluated  (if  evaluated)?  (Circle  all  that  apply.) 

Participants  completed  a survey 

Participants  were  interviewed  to  provide  feedback 

The  session  was  observed  by  an  evaluator 

My  classroom  was  observed 

Student  outcomes  in  my  classroom  were 

evaluated 

Some  other  form  of  evaluation  took 

place  (specify) 

No  discernible  evaluation  took  place 


a 

b 


c 

d 


e 

f 

g 


o 
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EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 


Section  IV:  Effectiveness  of  The  Mathematics  Related  Professional  Development  Activity 

The  following  questions  ask  about  the  effectiveness  of  the  same  professional  development  activity  you  described  in  Section  III. 

1 . To  what  extent  do  you  feel  that  your  knowledge  and  skills  have  been  enhanced  in  each  of  the  following  areas  as  a result  of  your 
participation  in  the  identified  professional  development  activity?  (Circle  one  response  on  each  line.) 


a. 

Curriculum  (e.g.,  units,  texts,  standards) 

Not 
at  All 
1 

2 

3 

4 

Great 

Extent 

5 

b. 

Instructional  methods 

1 

2 

3 

4 

5 

c. 

Approaches  to  assessment 

1 

2 

3 

4 

5 

d. 

Use  of  technology  in  instruction  (e.g., 
computers,  graphing  calculators) 

1 

2 

3 

4 

5 

e. 

Strategies  for  teaching  diverse  student 
populations  (e.g.,  students  with  disabilities, 
from  underrepresented  populations, 
economically  disadvantaged,  range 
of  abilities) 

1 

2 

3 

4 

5 

f. 

Deepening  knowledge  of  mathematics 

1 

2 

3 

4 

5 

9- 

Leadership  development 

1 

2 

3 

4 

5 

h. 

Adapting  teaching  to  meet  state 

assessment  requirements 

1 

2 

3 

4 

5 

i. 

Adapting  teaching  to  meet  state  standards  or  curriculum 
framework  requirements 

1 

2 

3 

4 

5 

j- 

Learning  about  state  assessments  in 
professional  development 

1 

2 

3 

4 

5 

k. 

Learning  about  state  standards  or 

curriculum  frameworks  in  professional 

development 

1 

2 

3 

4 

5 

1. 

Other:  ( please  specify)  

1 

2 

3 

4 

5 

2.  Have  you  attempted  to  introduce  changes  in  your  teaching  because  of  your  participation  in  the  identified  activity? 


Yes 1 

No 2 (skip  to  section  V) 
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EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 

4.  To  what  extent  have  you  made  each  of  the  following  changes  in  your  teaching  practices  as  a result  of  the  professional  development 
activity  (Circle  one  number  for  each  line): 


a. 

The  mathematics  curriculum 

No 

Chanae 

Minor 

Chanae 

Moderate 

Chanae 

Significant 

Chanae 

b. 

content 

The  cognitive  challenge  of 

0 

1 

2 

3 

c. 

math  classroom  activities 

The  instructional  methods  1 

0 

1 

2 

3 

d. 

employ 

The  types  or  mix  of  assessments  1 use 

0 

1 

2 

3 

e. 

to  evaluate  students..... 

The  ways  1 use  technology 
in  instruction  (calculator 

0 

1 

2 

3 

f. 

or  computer) 

The  approaches  1 take  to 

0 

1 

2 

3 

student  diversity 

0 

1 

2 

3 

Section  V:  Teacher  Background  And  Workload 


1 . Please  indicate  your  sex.  (Circle  one  response.). 


Female 1 

Male 2 


2.  Please  indicate  your  ethnicity/race.  (Circle  one  response.) 


American  Indian  or  Alaskan 

Native 

1 

Asian  or  Pacific  Islander 

2 

African  American,  not  of  Hispanic 

origin 

3 

White,  not  of  Hispanic  origin 

4 

Hispanic 

5 

Other:  (please  specify)  

6 

How 

many  years  of  teaching  experience  do  you  have? 

a. 

Overall 

years 

b. 

Teaching  math 

years 

c. 

Teaching  science 

years 

. E-24 
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EXHIBIT  E.3.21d  (Continued) 

Items  from  Teacher  Activity  Survey  Used  in  Analyses  Reported  in  Chapter  3 


4.  Please  mark  the  box[es]  next  to  the  degree(s)  you  hold.  Use  the  list  of  code  numbers  from  the  next  page  to  indicate  your  major  and 
minor  fields  of  study  for  each  degree.  (If  you  do  not  have  a second  major  or  minor  field,  please  enter  “00“.) 


Post-secondary  degrees 

(Circle  one  on 
each  line) 

Yes  No 

Enter  field  code  from  list  on  next 
page  (Question  5) 

Enter  year 
of  degree 

Major 

Field 

Second 

Major/Minor 

a.  Bachelor’s  Degree 

1 2 

b.  2nd  Bachelor’s  Degree 

1 2 

c.  Master’s  Degree 

1 2 

d.  2nd  Master’s  Degree 

1 2 

e.  Doctorate  (e.g.,  Ph.D.,  Ed.D.) 

1 2 

f.  Professional  Degree  (e.g.,  M.D., 
L.L.B.,  J.D.,  D.D.S.) 

1 2 

5.  Certification:  Circle  the  field  codes  of  the  subjects  listed  below  for  which  you  have  state 
certification. 

List  of  Teaching  Field  and  College  Major  Codes: 


Education 

Science 

Elementary  education 

01 

Biology/life  science 

11 

Middle  school  education 

02 

Geology/earth  sciences 

12 

Secondary  education 

03 

Chemistry 

13 

Mathematics  education 

04 

Physics 

14 

Science  education 

05 

Engineering 

15 

Special  education 

06 

Other  natural  sciences 

16 

Bilingual  education 

07 

Other  general  education 

08 

Math/ComDuter  Science 

Other 

Mathematics 

21 

English/language  arts 

31 

Computer  science 

22 

Social  sciences/social  studies 

32 

Vocational  education/agriculture 

33 

Arts/music 

34 

Foreign  languages 

35 

Philosophy 

36 

Psychology 

37 

Health/PE 

38 

Other:  (please  specify) 

40 

Administration 

50 
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APPENDIX  F 


SUPPLEMENT  FOR  CHAPTER  4 


This  appendix  provides  supplementary  information  for  the  analyses  reported  in  Chapter  4. 
Each  Exhibit  in  the  appendix  corresponds  to  an  Exhibit  of  the  same  number  in  Chapter  4.  The  boxes 
at  the  bottom  of  each  Exhibit  refer  to  District  Eisenhower  Coordinator  Telephone  Survey  item 
numbers  or  composite  variables  on  which  the  exhibit  is  based.  District  Coordinator  Survey  item 
numbers  begin  with  the  letter  “D,”  followed  by  the  section  of  the  survey  (part  A or  B)  and  the  item 
involved.  For  example,  item  numbers  “DA044_1,  DA044_2,  and  DA044_3”  refer  to  the  District 
Coordinator  Survey  part  A,  item  44_1,  2,  and  3.  (Item  44_1  is  shown  on  printed  copies  of  the  survey 
as  44a,  44_2  is  shown  as  44b,  etc.)  All  items  referred  to  in  the  chapter  appear  at  the  end  of  Appendix 
G,  in  Exhibit  5.1  Id.  Composite  variables,  which  appear  in  all  capital  letters  (for  example, 

PCTIDW),  are  defined  in  Exhibit  5.1  lb. 


EXHIBIT  F.4.1 

Percent  of  Teachers  in  Districts  Using  Eisenhower  Funds  to  Support  Professional 
Development  Activities,  by  Subject  Area  (n=353) 


Subject  Areas 

Percent 

Math 

98.74 

Science 

98.67 

Language  Arts 

43.87 

Social  Studies 

26.41 

Technology 

24.23 

Art  or  Music 

5.24 

Health  or  PE 

1.86 

Special  Education 

4.28 

Other  subjects 

11.75 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998,  reporting 
on  the  1997-1998  school  year. 

Variables : DA044J-3 
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EXHIBIT  F.4.2 


Percent  of  Participations  in  Eisenhower-assisted  Professional  Development 

Activities,  by  Subject  Area  (n=312) 


Subject  Areas 

Percent  of  Participations 

Math 

43.99 

Science 

30.22 

Technology 

3.56 

Combination  of  math,  science,  & technology 

11.43 

Other  subjects 

10.78 

Source : District  coordinator  lists  of  Eisenhower-assisted  activities  provided  in  the  district  from  July  1 
through  December  3 1 , 1997. 

Variables:  PCTACT 


EXHIBIT  F.4.3 

Percent  of  Participations  in  Traditional  Types  of  Eisenhower-assisted  Activities 

(n=353) 


Traditional  Types  of  Professional  Development 

Percent  of  Participations 

In-district  workshops  and  institutes 

55.88 

Out-of-district  workshops  and  conferences 

20.64 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables : PCTIDW,  PCTODW 

454 

o 

ERIC 


F-2 


EXHIBIT  F.4.4a 


Percent  of  Eisenhower-assisted  In-district  Workshops  and  Institutes, 

by  Contact  Hours  (n=3141) 

& 

EXHIBIT  F.4.4b 

Percent  of  Eisenhower-assisted  In-district  Workshops  and  Institutes, 

by  Time  Span  (n=314) 


Duration  Percent 


Total  Contact  Hours 

Less  than  4 hours 
4-8  hours 
9-40  hours 
More  than  40  hours 

17.95 

34.74 

38.31 

8.98 

Span  Over  Time 

One  day 

31.74 

2-7  days 

36.59 

8 days  to  one  month 

8.08 

One  month  to  one  year 

16.25 

More  than  one  year 

7.32 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998, 
Variables : DA083J-4,  DA084J-5 


1 Of  our  total  sample  of  363  district  Eisenhower  coordinators,  314  provided  Eisenhower- assisted  in-district 
workshops  and  institutes;  therefore  all  analyses  that  pertain  to  in-district  workshops  and  institutes  have  a sample  of 


314. 
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EXHIBIT  F.4.5a 


Percent  of  Teachers  in  Districts  in  which  Eisenhower-assisted  In-district  Workshops 
and  Institutes  Include  Collective  Participation  (n=314) 


Collective  Participation  Percent 

All  T eachers  in  Department  or  Grade  74.35 

All  Teachers  in  School 58.19 

Source-.  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables.  DA078_3,  DA078_4  if  DA072=1 


EXHIBIT  F.4.5b 

Collective  Participation  in  Eisenhower-assisted  In-district  Workshops  and  Institutes, 
Overall  and  by  District  Poverty  and  District  Size  (n=314) 


Mean 

SD 

n 

F df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p < .05) 

Overall 

.66 

.38 

314 

Poverty  Level 

.77  2,302 

.46 

Low  Medium  High 

Low 

.64 

(.37) 

97 

Medium 

.65 

(.37) 

119 

High 

.70 

(.39) 

98 

District  Size 

6.52  3,302 

.00 

Small  Medium  Large  Consortium 

Small 

.55 

(.36) 

68 

X 

Medium 

.62 

(.37) 

116 

X X 

Large 

.77 

(.37) 

96 

Consortium 

.71 

(.39) 

34 

Poverty*Size 

1.82  6,302 

.09 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables : sch-base 
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EXHIBIT  F.4.6 

Percent  of  Teachers  in  Districts  that  Provide  Each  of  Five  Types  of  Opportunities  for 
Active  Learning  in  Eisenhower-assisted  In-district  Workshops  and  Institutes  (n=314) 


Types  of  active  learning  Percent 

Observe  professional  development  providers  demonstrating  or 

modeling  skills  96.06 

Observe  other  teachers  teaching  74.22 

Practice  under  simulated  conditions  with  feedback  69.36 

Meet  in  groups  to  discuss  problems  in  implementation  96.80 

Observed  teaching  in  his  or  her  own  classroom 49.10 


Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables:  DAQ81J,  DAQ81__2,  DAQ81_3,  DAQ81_4,  DAQ81_5  where  DA07I  = 1 and  DAQ72=1 


EXHIBIT  F.4.7 

Number  of  Types  of  Opportunities  for  Active  Learning  in  Eisenhower-assisted  In- 
District  Workshops  and  Institutes,  Overall  and  by  District  Poverty  and  District  Size 

(n=314) 


Significant  Tukey  Pairwise  Contrasts 


Mean 

SD 

n 

F 

df 

P 

(where  p<. 05) 

Overall 

3.83 

(1.10) 

314 

Poverty  Level 

1.03 

2, 302 

0.35 

Low 

Medium  High 

Low 

3.70 

(0.99) 

108 

Medium 

3.81 

(1.13) 

118 

High 

4.07 

(1.11) 

88 

District  Size 

11.10 

3, 302 

0.00 

Small 

Medium  Large  Consortium 

Small 

3.29 

(1.15) 

68 

Medium 

3.76 

(0.98) 

116 

X 

Large 

4.29 

(0.94) 

96 

X 

X 

Consortium 

3.94 

(1.16) 

34 

X 

Poverty*Size 

.37 

6, 302 

0.90 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  STCH1DW 
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EXHIBIT  F.4.8a 


Percent  of  Teachers  in  Districts  That  Support  Reform  Types  of 
Professional  Development  Activities  with  Eisenhower  Funds  (n=353) 

& 

EXHIBIT  F.4.8b 


Percent  of  Participations  in  Reform  Types  of  Eisenhower-assisted  Professional 

Development  Activities  (n=353) 


Percent  of  teachers  in  districts 
that  use  Eisenhower  funds  to 
support  reform  types  of 
professional  development 
activities 


Percent  of  participations  in  reform 
types  of  Eisenhower-assisted 
activities 


Study  Group 

20.55 

5.43 

Teacher  Network 

37.49 

4.49 

Mentoring 

30.22 

2.51 

Committee  or  Task  Force 

35.42 

3.48 

Internship 

3.57 

.18 

Individual  Research  Project 

6.54 

.24 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables : 8a.  HASSTD,  HASNET,  HASMNT,  HASCOM,  HASRCT,  HASCRS,  HASINT,  HASRES  where  NUMTYPER 
ne  0;  8b.  PCTSTD,  PCTNET,  PCTMNT,  PCTCOM,  PCTRCT,  PCTCRS,  PCTINT,  PCTRES 
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EXHIBIT  F.4.9a 


Percent  of  Participations  in  Reform  Types  of  Eisenhower-assisted  Professional 
Development  Activities,  Overall  and  by  District  Poverty  and  District  Size  (n=353) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

Overall 

16.33 

(6.71) 

353 

Poverty  Level 

5.57 

2,341 

0.00 

Low 

. Medium 

High 

Low 

13.96 

(19.94) 

119 

Medium 

19.78 

(22.35) 

128 

High 

14.95 

(20.88) 

106 

District  Size 

1.24 

3, 341 

0.29 

Small 

Medium 

Large  Consortium 

Small 

13.31 

(23.86) 

90 

Medium 

16.33 

(19.61) 

128 

Large 

19.34 

(21.83) 

98 

Consortium 

14.83 

(17.41) 

37 

Poverty'Size 

2.23 

6,341 

0.04 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  new 3 


EXHIBIT  F.4.9b 

Percent  of  Teacher  Participations  in  Reform  Types  of  Professional  Development 
Activities,  Interaction  of  District  Poverty  and  District  Size  (n=353) 

High  Poverty  Medium  Poverty  Low  Poverty 


District  Size 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Small 

6.18 

(14.95) 

19.40 

(26.76) 

11.85 

(24.48) 

Medium 

13.54 

(17.55) 

15.55 

(19.73) 

18.82 

(20.81) 

Large 

20.94 

(24.71) 

24.34 

(22.35) 

8.84 

(8.43) 

Consortium 

6.47 

(12.79) 

22.88 

(19.42) 

9.39 

(10.84) 

| Source:  Telephone  Survey  of  District  Eisenhower  Coordinators.  Spring  1998. 
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EXHIBIT  F.4.10 


Number  of  Types  of  Eisenhower-assisted  Activities,  Overall  and  by  District  Poverty 

and  District  Size  (n=353) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 
(where  p<. 05) 

Overall 

3.41 

(1.67) 

353 

Poverty  Level 

4.37 

2, 341 

0.00 

Low 

Medium  High 

Low 

3.02 

(1.55) 

119 

Medium 

3.63 

(1.58) 

128 

X 

High 

3.69 

(1.85) 

106 

X 

District  Size 

26.23 

3, 341 

0.00 

Small 

Medium  Large  Consortium 

Small 

2.42 

(1.31) 

90 

Medium 

3.23 

(1.29) 

128 

X 

Large 

4.39 

(1.80) 

98 

X 

X 

Consortium 

3.85 

(1.80) 

37 

X 

Poverty*Size 

.87 

6, 341 

.51 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1 998. 
Variables : NUMTYPEF 


EXHIBIT  F.4.11a 

Percent  of  Teachers  in  Districts  by  Span  of  Eisenhower-assisted  Reform  Activities 

(n  varies  by  type) 


Types  of 
reform  activities 

Percent  of  teachers  in 
districts  with  typical 
activities  lasting  less 
than  one  month 

Percent  of  teachers  in 
districts  with  typical 
activities  lasting  from  one 
month  to  one  year 

Percent  of  teachers  in 
districts  with  typical 
activities  lasting  more 
than  a year 

Study  groups 

44.48 

39.49 

16.02 

Teacher  Networks 

25.09 

46.25 

28.66 

Mentoring 

24.95 

53.84 

21.21 

Committee  or  task  force 

42.91 

44.67 

12.41 

Internships 

55.55 

44.44 

0.0 

Individual  research 

25.00 

58.33 

16.67 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables : DURINTN,  DURMNTN,  DURCOMN,  DURSTDN,  DURRESN,  DURNETN 
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EXHIBIT  F.4.1 1b 


Average  Span  of  Eisenhower-assisted  Activities,  Overall  and  by  District  Poverty  and 

District  Size  (n=353) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

Overall 
Poverty  Level 

1.39 

(.47) 

353 

1.63 

2, 341 

0.20 

Low 

Medium  High 

Low 

1.36 

(.47) 

119 

Medium 

1.42 

(.48) 

128 

High 

1.40 

(.46) 

106 

District  Size 

3.43 

3,341 

0.02 

Small 

Medium  Large  Consortium 

Small 

1.28 

(.43) 

90 

Medium 

1.39 

(.49) 

128 

Large 

1.50 

(.45) 

98 

X 

Consortium 

1.40 

(.49) 

37 

Poverty'Size 

1.20 

6,341 

.30 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables : DURATION 

EXHIBIT  F.4.1 2 

Percent  of  Teachers  in  Districts  That  Fund  Workshops  with  Eisenhower  Funds,  Other 

Sources,  or  Not  At  All  (n=353) 


Types  of 

traditional  activities 

Percent  of  teachers 
in  districts  that  fund 
workshops  with 
Eisenhower  funds 

Percent  of  teachers  in 
districts  that  fund 
workshops  without 
Eisenhower  funds 

Percent  of  teachers  in 
districts  that  do  not 
offer  workshops  at  all 

In-district  workshops 

89.18 

3.43 

7.33 

Out-of-district  workshops 

81.73 

7.47 

10.80 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  DA07I,  DA072  DA085,  DA086 

EXHIBIT  F.4.13 


Percent  of  Teachers  in  Districts  That  Fund  Reform  Activities  with  Eisenhower  Funds, 

Other  Sources,  or  Not  at  All  (n=353) 


Types  of 
reform  activities 

Percent  of  teachers  in 
districts  that  offer 
activity  and  support  it 
with  Eisenhower  funds 

Percent  of  teachers  in 
districts  that  offer  activity 
and  support  it  without 
Eisenhower  funds 

Percent  of  teachers  in 
districts  that  do  not  offer 
activity  at  all 

Study  groups 

20.55 

27.86 

51.59 

Teacher  networks 

37.49 

23.96 

38.54 

Mentoring 

30.22 

31.49 

38.29 

Committee  or  task  force 

35.42 

35.37 

29.21 

Internships 

3.57 

16.71 

79.71 

Individual  research 

6.54 

18.92 

74.54 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables : DA099,  DA100,  DA107,  DA108,  DAI  14,  DAI  IS.  DA127,  DA128,  DA135,  DA  06,  DA142,  DA143 


EXHIBIT  F.4.14 

Percent  of  Teachers  in  Districts  that  Report  Placing  No,  Some,  or  a Strong  Emphasis 
on  Recruiting  Teachers  of  Special  Student  Populations  (n=363) 


Special  populations  of  teachers 

No  emphasis 

Some  emphasis 

Strong  emphasis 

Teachers  in  Title  1 schools 

41.58 

28.21 

30.21 

Special  education  teachers 

40.69 

38.64 

20.67 

Limited  English 

53.49 

26.87 

19.63 

Low  achievement  schools 

33.36 

23.16 

43.48 

High  poverty  schools 

42.38 

17.95 

39.66 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables : DB068,  DB069,  DB070,  DB071.  DB072 
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EXHIBIT  F.4.15 


Correlation  of  the  Emphasis  on  Recruitment  of  Teachers  of  Special  Populations  of 

Students  (n=363) 


Special  populations  of 
teachers 

Teachers  in  Title  1 
schools 

Special  Education 

Limited  English 
Proficient 

Low  Achievement 

Special  education  teachers 

.65*** 

Limited  English 

.55*** 

.62*** 

Low  achievement  schools 

.63*** 

.59*** 

.53*** 

High  poverty  schools 

.68*** 

.55*** 

.59*** 

.76*** 

***p<.001 


Source.  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables.  DB068,  DB069,  DB070,  DB071,  DB072 


EXHIBIT  F.4.16 

Extent  of  Recruitment  of  Teachers  of  Special  Populations  of  Students,  Overall  and  by 

District  Poverty  and  District  Size  (n=363) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 
(where  p<. 05) 

Overall 

1.88 

(22.03) 

363 

Poverty  Level 

14.56 

2, 351 

0.00 

Low 

Medium  High 

Low 

1.70 

(.61) 

124 

Medium 

1.83 

(.66) 

131 

High 

2.18 

(.72) 

108 

X 

X 

District  Size 

6.75 

3, 351 

0.00 

Small 

Medium  Large  Consortium 

Small 

1.64 

(.64) 

98 

- 

Medium 

1.85 

(.69) 

130 

Large 

2.15 

(.66) 

98 

X 

X 

Consortium 

1.84 

(.74) 

37 

Poverty*Size 

.87 

6, 351 

.51 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables'.  Special 
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EXHIBIT  F.4.17 


Percent  of  Participations  in  District  Eisenhower-assisted  Activities  by  How  Teachers 

Come  to  Participate  (n=3582) 


Methods  of  Participation 

Percent  of  Participations 

Volunteer 

67.68 

Principal  Selected 

24.21 

Provider  Selected 

2.28 

Rotation 

4.96 

Other  Ways 

3.72 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables : DA063J,  DA063_2.  DA063_3,  DA063_4,  DA063_5 


EXHIBIT  F.4.18 

Percent  of  Teachers  in  Districts  That  Use  Various  Strategies  to  Increase  the 
Participation  of  Teachers  in  Eisenhower-assisted  Activities  (n=363) 


Strategies  to  increase  Participation 

Percent 

Publicize 

95.79 

Tailor  the  focus  towards  the  needs  of  special  populations 

79.72 

Use  incentives 

71.90 

Other  strategies 

22.92 

No  special  strategies  are  used 

0.80 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables : DB075J,  DB075_2,  DB075_3,  DB075_4,  DB075_5 


2 Of  our  total  sample  of  363  district  Eisenhower  coordinators,  five  did  not  answer  this  survey  question;  therefore  the 

sample  size  for  this  variable  was  358.  464 
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APPENDIX  G 


SUPPLEMENT  FOR  CHAPTER  5 


This  appendix  provides  supplementary  information  for  the  analyses  reported  in  Chapter  5. 
Each  Exhibit  in  the  appendix  corresponds  to  an  Exhibit  of  the  same  number  in  Chapter  5.  The  boxes 
at  the  bottom  of  each  Exhibit  refer  to  District  Eisenhower  Coordinator  Telephone  Survey  item 
numbers  or  composite  variables  on  which  the  exhibit  is  based.  District  Coordinator  Survey  item 
numbers  begin  with  the  letter  “D,”  followed  by  the  section  of  the  survey  (part  A or  B)  and  the  item 
involved.  For  example,  item  numbers  “DA044_1,  DA044_2,  and  DA044_3”  refer  to  the  District 
Coordinator  Survey  part  A,  item  44_1,  2,  and  3.  (Item  44_1  is  shown  on  printed  copies  of  the  survey 
as  44a,  44_2  is  shown  as  44b,  etc.)  All  items  referred  to  in  the  chapter  appear  in  Exhibit  5. 1 Id. 
Composite  variables,  which  appear  in  all  capital  letters  (for  example,  PDIRT),  are  defined  in  Exhibit 
5.11b. 


EXHIBIT  G.5.1a 

Percent  of  Teachers  in  Districts  in  which  State  and  District  Mathematics  and  Science 

Standards  and  Assessments  Exist  (n=363) 


State 

State 

District 

District 

Standards 

Assessments 

Standards 

Assessments 

Mathematics 

90.8 

91.3 

84.9 

69.5 

Science 

84.6 

72.3 

78.5 

42.0 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  DA  1 54.DA  1 56,  DA  1 59,  DA  1 6 1 , DA  1 65,  DA  1 67,  DA  1 70,  DA  1 72 


EXHIBIT  G.5.1b 

Percent  of  Teachers  in  Districts  Where  Eisenhower-assisted  Activities  Are  Aligned 
“To  a Large  Extent”  with  State  and/or  District  Standards  and/or  Assessments  (Where 
Such  Standards  and  Assessments  Exist)  (n  varies) 


State 

Standards 

State 

Assessments 

District 

Standards 

District 

Assessments 

Mathematics 

85.1 

70.8 

85.4 

69.2 

Science 

83.4 

70.4 

85.7 

66.8 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  DA155,  DA157,  DA160,  DA162,  DA166,  DA168,  DA171,  DA173 
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EXHIBIT  G.5.1C 

Degree  of  Alignment  Between  Eisenhower-assisted  Activities  and  Standards  and 
Assessments,  Overall  and  by  District  Poverty  and  District  Size  (n=363) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

Overall 

4.0 

.99 

363 

Poverty  Level 

10 

2,  351 

.90 

Low  Medium  High 

Low 

3.92 

.99 

124 

Medium 

3.98 

.96 

131 

High 

4.11 

1.01 

108 

District  Size 

4.26 

3,  351 

.01 

Small  Medium  Large  Consortium 

Small 

3.74 

1.10 

98 

Medium 

4.03 

.91 

130 

Large 

4.25 

.87 

98 

X 

Consortium 

3.90 

1.10 

37 

Poverty*Size 

.79 

6, 351 

.58 

1 Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
| Variables,  pdirtx 
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EXHIBIT  G.5.2 


Percent  of  Teachers  in  Districts  According  to  Eisenhower  Coordinator’s  Roles  within 

the  District  Office  (n=363) 


Position  within  district  office 

Percent  of  teachers  in 
districts  in  which  the 
district  has  the 
position 

Percent  of  teachers  in 
districts  in  which 
district  Eisenhower 
Coordinator  fills 
position 

Percent  of  teachers  in 
districts  in  which  the 
person  in  the  position 
participates  in 
Eisenhower  decision 
making 

General  Curriculum/lnstruction 

80.2 

48.0 

90.5 

Coordinator 

Mathematics  Coordinator 

58.0 

46.4 

96.3 

Science  Coordinator 

56.8 

46.7 

96.9 

Professional  Development 

69.1 

38.9 

80.6 

Coordinator 

Special  Education  Coordinator 

91.5 

8.4 

61.9 

Title  1 Coordinator 

86.6 

24.7 

78.1 

Federal  Programs  Coordinator 

59.5 

42.6 

82.8 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables:  DA009-DA040 

Note:  There  may  be  overlap  in  the  positions  filled  by  Eisenhower  coordinators  or  positions  filled  by  other  individuals.  For  instance, 
an  Eisenhower  coordinator  may  be  the  district’s  curriculum  coordinator  and  the  district’s  mathematics  and  science  coordinator. 
Similarly,  any  other  individual  who  works  closely  with  the  Eisenhower  coordinator  may  serve  in  multiple  roles,  for  instance  as  the 
Title  I coordinator  and  the  federal  programs  coordinator. 


46 


i 


G-3 


EXHIBIT  G.5.3a 

Percent  of  Teachers  in  Districts  in  which  Eisenhower  Activities  Coordinate  (Co-fund 
and/or  Work  Closely)  with  Other  Programs  (n  varies) 

Federal  Program 

Percent  of  teachers  in 
districts  where  federal 
program  operates  in 
state/district 

Percent  of  teachers  in 
districts  where  federal 
program  supports 
professional 
development  in  district 

In  districts  where 
federal  program 
operates  and  supports 
professional 
development, 
percent  of  teachers  in 
districts  that  co-fund 
with  other  programs 

Percent  of  teachers 
in  districts  with 
other  federal 
programs  in  which 
Eisenhower 
coordinator  works 
closely  with 
other  programs 

SSI 

44.4 

66.7 

65.7 

59.2 

USI 

17.0 

89.5 

86.1 

85.9 

RSI 

3.7 

78.3 

27.7 

85.5 

LSC 

12.0 

91.1 

75.7 

91.9 

Title  1,  A 

90.9 

90.6 

49.9 

81.8 

Title  1,  C 

39.6 

62.6 

35.1 

66.7 

Title  III 

28.7 

90.8 

39.3 

73.4 

Title  VI 

77.0 

72.8 

47.7 

77.9 

Title  VII 

46.6 

82.0 

27.6 

60.7 

Title  IX 

19.1 

56.9 

4.9 

42.3 

IDEA 

76.6 

85.2 

21.8 

66.5 

Goals  2000 

85.8 

73.5 

33.9 

61.0 

School  to  Work 

76.8 

85.5 

27.6 

65.5 

Perkins 

73.8 

83.5 

18.7 

62.2 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables : DB005-DB060 

Note.  Results  on  co-funding  for  each  listed  program  are  based  on  districts  that  participate  in  the  program  and  in  which  the  program 
supports  professional  development.  Results  for  working  closely  with  each  program  are  based  on  districts  that  participate  in  the 
program. 
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EXHIBIT  G.5.3b 


Extent  of  Co-Funding  of  Eisenhower-assisted  Activities  with  Those  of  Other  Federal 
Programs,  Overall  and  by  District  Poverty  and  District  Size  (n=363) 


Significant  Tukey  Pairwise  Contrasts 


Mean 

SD 

n 

F 

df 

P 

(where  p<. 05) 

Overall 

2.06 

(2.07) 

363 

Poverty  Level 

15.42 

2, 351 

0.00 

Low  Medium  High 

Low 

1.19 

(1.52) 

124 

Medium 

1.95 

(1.94) 

131 

Significant  interaction  effects 

High 

3.00 

(2.28) 

108 

District  Size 

11.79 

3, 351 

0.00 

Small  Medium  Large  Consortium 

Small 

1.10 

(1.32) 

98 

Medium 

1.81 

(1.75) 

130 

Large 

2.97 

(2.35) 

98 

Significant  interaction  effects 

Consortium 

2.45 

(2.55) 

37 

Poverty*Size 

2.20 

6, 351 

0.04 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  NPROGF  


EXHIBIT  G.5.3C 

Extent  of  Co-funding  of  Eisenhower-assisted  Activities  with  Those  of  Other  Federal 
Programs,  Interaction  of  District  Poverty  and  District  Size  (n=363) 


High  Poverty  Medium  Poverty  Low  Poverty 


District  Size 

Mean 

SD 

Mean 

SD 

Mean  SD 

Small 

1.70 

(1.52) 

1.43 

(1.45) 

.48  (.73) 

Medium 

2.41 

(1.85) 

1.87 

(1.91) 

1.40  (1.43) 

Large 

3.70 

(2.36) 

2.87 

(2.26) 

1.50  (1.68) 

Consortium 

4.06 

(3.07) 

1.50 

(1.78) 

2.40  (2.62) 

| Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

1 

G-5 


EXHIBIT  G.5.4 


Percent  of  Teachers  in  Districts  According  to  Status  of  District  Performance 
Indicators  for  Professional  Development  (n=363) 


Status  of  district  performance  indicators 

Percent  of  teachers  in  districts 

Developed  indicators 

32.00 

Developing  indicators 

36.00 

Not  developing  indicators 

32.00 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables : DB088  DB089 

EXHIBIT  G.5.5 

Percent  of  Teachers  in  Districts  that  Use  Different  Methods  to  Assess  Teachers’ 
Professional  Development  Needs  (n=363) 


Needs  assessment  method Percent 


Teacher  survey  79.70 

Teacher  meetings  69.90 

Principal  survey  70.60 

Student  performance  64.60 

Informal  conversation  75.70 


Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  DB079J,  DB079_2,  DB079_3,  DB079_4,  DB079_5 
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EXHIBIT  G.5.6 


Percent  of  Teachers  in  Districts  that  Use  Different  Methods  to  Evaluate  Eisenhower- 

assisted  Activities  (n=363) 


Evaluation  techniques 

Percent 

Teacher  participation 

69.90 

Teacher  survey 

84.60 

Observe  teachers 

71.40 

Student  achievement  scores 

60.00 

Source.  Telephone  Survey  of  District  Eisenhower  Coordinators.  Spring  1998. 
Variables : DB084_1— 4 


EXHIBIT  G.5.7 

Percent  of  Teachers  in  Districts  Whose  Eisenhower  Staff  Provide  Different  Types  of 
Guidance  about  Professional  Development  to  Schools  and  Professional 

Development  Providers  (n=363) 


Types  of  guidance 

Percent  of  teachers  in  districts 
providing  guidance  to  schools 

Percent  of  teachers  in  districts 
providing  guidance  to  providers 

Classroom  visits 

■ 64.06 

53.08 

Phone  calls 

78.86 

74.72 

Required  reports 

32.55 

26.84 

Required  evaluations 

40.64 

44.11 

Interpreting  Title  II  rules 

63.11 

39.83 

Conduct  needs  assessments 

76.65 

46.79 

Develop  plans 

81.08 

51.21 

Develop  activities 

82.23 

56.02 

Develop  indicators 

39.66 

25.64 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 

Variables'.  DB143J  - DB143_4,  DBI45_1  - DB145_5,  DB147_1-DB147_4,  DB  149,1-DB  149_5 
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EXHIBIT  G.5.8 


Extent  of  District  Continuous  Improvement  Efforts,  Overall  and  by  District  Poverty 

and  District  Size  (n=363) 


Mean 

SD 

n 

F 

df 

P 

Tukey  Pairwise  Contrasts 

Overall 

2.64 

(1.08) 

363 

Poverty  Level 

4.17 

2,351 

0.01 

Low 

Medium  High 

Low 

2.46 

(1.10) 

136 

Medium 

2.60 

(1.02) 

130 

High 

2.86 

(1.09) 

97 

X 

District  Size 

19.36 

3,  351 

0.00 

Small 

Medium  Large  Consortium 

Small 

1.97 

(0.99) 

98 

Medium 

2.68 

(0.97) 

130 

X 

Large 

3.06 

(0.97) 

98 

X 

X 

Consortium 

2.96 

(1.17) 

37 

X 

Poverty*Size 

2.06 

6,  351 

.06 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables.  CONIMP.D 

Scale  Reliability:  .61 


EXHIBIT  G.5.9a 


Percent  of  Teachers  in  Districts  Reporting  That  None,  Some,  Most,  or  All 
Professional  Development  Activities  Are  Planned  at  the  District,  School,  and  Cluster 

Levels  (n=363) 


Percentage  of  teachers 
in  districts  in  which 
activities  are  planned 
at  the  indicated  level 
none  of  the  time 

Percentage  of 
teachers  in  districts 
in  which  activities 
are  planned  at  the 
indicated  level 
some  of  the  time 

Percentage  of 
teachers  in  districts 
in  which  activities 
are  planned  at  the 
indicated  level  most 
of  the  time 

Percentage  of 
teachers  in  districts 
in  which  activities 
are  planned  at  the 
indicated  level  all  of 
the  time 

District  Level 

6.6 

43.1 

34.0 

16.2 

School  Level 

19.2 

48.1 

22.7 

9.7 

Cluster  Level 

56.5 

36.2 

5.3 

2.0 

Source : Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables.  DA050,  DA060,  DA057 
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EXHIBIT  G.5.9b 


Extent  to  Which  Professional  Development  Activities  Are  Planned  at  the  School  vs. 
District  Level,  Overall  and  by  District  Poverty  and  District  Size  (n=363) 


Mean 

SO 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

Overall 

2.32 

(0.67) 

363 

Poverty  Level 

1.49 

2,  351 

0.23 

Low 

Medium 

High 

Low 

2.25 

(0.65) 

136 

Medium 

2.39 

(0.68) 

130 

High 

2.28 

(0.70) 

97 

District  Size 

4.71 

3, 351 

0.00 

Small 

Medium 

Large  Consortium 

Small 

2.44 

(0.76) 

98 

X 

Medium 

2.32 

(0.62) 

130 

X 

Large 

2.30 

(0.67) 

98 

Consortium 

1.99 

(0.56) 

37 

Poverty'Size 

0.62 

6, 351 

0.71 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  DLEVPLAN 


EXHIBIT  G.5.10 


Percent  of  Teachers  in  Districts  with  Different  Types  of  Teacher  Involvement  in 
School-  and  Cluster-level  Professional  Development  Planning  (n=363) 


Percent  of  teachers  in  districts 
with  different  types  of  teacher 
involvement  in  school-level 
professional  development 

Percent  of  teachers  in  districts 
with  different  types  of  teacher 
involvement  in  cluster-level 
professional  development 

Lead  teachers/department  chairs 

76.8 

40.2 

Teacher  committees 

61.6 

29.6 

Individual  teachers 

68.7 

33.4 

Source:  Telephone  Survey  of  District  Eisenhower  Coordinators,  Spring  1998. 
Variables:  DA058-2.3.4  and  DA06I -2.3.4 
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EXHIBIT  G.5.1  la  (Continued) 

Relationship  of  District  Implementation  to  Design  Characteristics 
of  Professional  Development 
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EXHIBIT  G.5.11C  (Continued) 

Correlations  among  District  Professional  Development  Variables 
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EXHIBIT  G.5.11d 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 

DISTRICT  TELEPHONE  INTERVIEW 

NOTE:  The  District  Telephone  Interview  was  administered  as  a Computer-Assisted  form,  and  the  items  as  administered  appeared  on 
a computer  screen.  The  following  printed  items  parallel  the  versions  that  appeared  on  the  CATI  screen,  but  some  details  have  been 
changed. 

PART  A 

Organizational  structure 

5-38.  I’d  like  to  ask  you  first  about  the  structure  of  your  district  office  and  your  place  in  it. 


Does  your 
district  have 
position? 

Is  it  funded  wholly 
or  in  part  by 
Eisenhower 
funds? 

Do  you  fill  the  role? 

Does  the  person  in 
the  position  work 
with  you  in  making 
decisions  about  the 
Eisenhower 
program? 

Eisenhower  coordinator 

Y N 

Y N 

YN 

Curriculum/instruction  coordinator 

Y N 

Y N 

Y N 

Y N 

Mathematics  coordinator 

Y N 

Y N 

Y N 

Y N 

Science  coordinator 

Y N 

Y N 

Y N 

Y N 

Professional  development 
coordinator 

Y N 

Y N 

Y N 

Y N 

Special  education  coordinator 

Y N 

Y N 

Y N 

Y N 

Title  1 coordinator 

Y N 

Y N 

Y N 

Y N 

Federal  programs  coordinator 

Y N 

Y N 

Y N 

Y N 

Are  there  other  district  office  staff 
who  work  with  you  in  making 
decisions  about  Eisenhower? 
(specify) 

Y N 

Y N 

Professional  development 

44.  In  which  of  the  following  subject  areas  do  you  support  professional  development  using  Eisenhower  funds?  (check  all  that  apply) 

a.  Mathematics? 

b.  Science? 

c.  Any  Other  (specify) 

50.  How  many  of  your  district’s  Eisenhower-supported  activities  are  planned  at  the  DISTRICT  LEVEL? 

0 None 

1 Some 

2 Most 

3 All 


G-17 


EXHIBIT  G.5.1  Id  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


51 . Which  of  the  following  best  describes  the  role  teachers  play  in  making  decisions  about  Eisenhower-supported  activities  planned 
at  the  district  level? 

a.  Do  teachers  participate  in  a formal  planning  committee 

b.  Are  teachers  consulted  informally 

c.  Are  teachers  consulted  in  a needs  assessment  or 

d.  Do  teachers  not  play  a regular  role 

e.  Other  (specify) 

56.  Does  your  district  have  sub-districts,  clusters,  or  groups  of  schools  that  jointly  plan  and  administer  professional  development? 

1 Yes 

2 No  (skip  to  60) 

57.  How  many  of  your  district’s  Eisenhower-supported  activities  are  planned  at  the  CLUSTER  LEVEL? 

0 None 

1 Some 

2 Most 

3 All 

58.  Which  of  the  following  people  participate  in  making  decisions  about  Eisenhower-supported  activities  planned  at  the  cluster  level? 

a.  Do  principals? 

b.  Do  lead  teachers,  resource  teachers,  or  department  chairs? 

c.  Do  classroom  teachers,  through  a formally  organized  committee? 

d.  Do  teachers  as  individuals? 

e.  Do  others  (specify)? 

60.  How  many  of  your  district’s  Eisenhower-supported  activities  are  planned  at  the  SCHOOL  LEVEL? 

0 None 

1 Some 

2 Most 

3 All 

61 . Which  of  the  following  people  participate  in  making  decisions  about  Eisenhower-supported  activities  planned  at  the  school  level? 

a.  Do  principals? 

b.  Do  lead  teachers,  resource  teachers,  or  department  chairs? 

c.  Do  classroom  teachers,  through  a formally  organized  committee? 

d.  Do  teachers  as  individuals? 

e.  Do  others  (specify)? 

63.  Of  the  teachers  in  Eisenhower  activities,  what  percent  come  to  participate  in  each  of  the  following  ways? 

a.  What  percent  volunteer? 

b.  What  percent  are  selected  by  their  principal  or  other  administrator? 

c.  What  percent  are  selected  by  providers? 

d.  What  percent  participate  by  rotation?  » 

e.  What  percent  come  to  participate  in  other  ways? 
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EXHIBIT  G.5.1  Id  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


In-district  workshops  or  institutes 

71 . Did  your  district  support  in-district  workshops  or  institutes  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  85) 

72.  Were  any  of  your  workshops  or  institutes  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  85) 

76.  How  many  participants  attended,  counting  participants  more  than  once  if  they  attended  multiple  workshops/institutes? 

78.  For  which  of  the  following  groups  were  workshops  specifically  designed?  (circle  all  that  apply) 

Were  they  designed.... 

a.  for  teachers  as  individuals 

b.  for  teachers  as  representatives  of  their  departments,  grade  level,  or  school 

c.  for  all  teachers  in  department  or  grade-level  groupings 

d.  for  all  teachers  in  a school  or  set  of  schools 

e.  in  other  configurations?  (specify) . 

81 . Which  of  the  following  strategies  to  help  teachers  implement  new  skills,  if  any,  were  used  regularly  in  conjunction  with  Eisenhower 
workshops  or  institutes?  (circle  all  that  apply) 

a.  Did  teachers  observe  leaders  demonstrating  or  modeling  skills 

b.  Did  teachers  observe  other  teachers  teaching 

c.  Did  teachers  practice  under  simulated  conditions,  with  feedback 

d.  Did  teachers  meet  in  groups  and  discuss  problems  in  implementation 

e.  Were  teachers  observed  teaching  a regular  class? 

f.  Other  (please  specify) 

g.  No  strategies  were  used 

83.  Including  follow-up  events,  approximately  what  percent  of  your  Eisenhower-supported  workshops  or  institutes  lasted: 

a.  Less  than  4 hours  % 

b.  Between  4 - 8 hours  % 

c.  Between  9 - 40  hours  % 

d.  More  than  40  hours  % 

84.  Including  follow-up  activities,  approximately  what  percent  of  these  workshops  or  institutes  were: 


a.  Spread  over  one  day  % 

b.  Spread  over  two  to  seven  days  % 

c.  Spread  over  eight  days  to  one  month  % 

d.  Spread  over  more  than  one  month  to  one  year  % 

e.  Spread  over  more  than  a year  % 


EXHIBIT  G.5.1  Id  (Continued) 


Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


Out-of-district  workshops  or  institutes 

85.  Did  your  district  support  out-of-district  workshops  or  institutes  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  95) 

86.  Did  you  use  Eisenhower  funds  to  support  teachers  in  attending  out-of-district  workshops  or  conferences? 

1 Yes 

2 No  (skip  to  95) 

88.  How  many  teachers  did  Eisenhower  funds  support  to  attend  out-of-district  workshops  or  conferences  from  July  1 through 
December  1 997? 


College  courses 

95.  Did  your  district  support  attendance  at  college  courses  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  99) 

96.  Did  you  use  Eisenhower  funds  to  support  teachers  in  taking  college  courses? 

1 Yes 

2 No  (skip  to  99) 

98.  How  many  teachers  did  you  support  for  college  courses  with  Eisenhower  funds  from  July  1 through  December  1 997? ' 


Teacher  collaboratives  or  networks 

99.  Did  your  district  support  participation  in  teacher  collaboratives  or  networks  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  107) 

100.  Were  any  teacher  collaboratives  or  networks  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  107) 

102.  How  many  teachers  did  Eisenhower  funds  support  to  participate  in  such  collaboratives  or  networks  from  July  1 through 
December  1997? 

106.  Over  what  time  period  did  the  typical  teacher  participate  in  these  networks? 

a.  One  month  or  less 

b.  More  than  one  month  but  less  than  six  months 

c.  Between  six  months  and  one  year,  or 

d.  More  than  one  year 


Internship  or  immersion  activities 

107.  Did  your  district  support  internship  or  immersion  activities  from  July 

1 Yes 

2 No  (skip  to  114) 


through  December  1977? 
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EXHBIT  G.5.1  Id  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


108.  Were  any  internship  or  immersion  activities  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  114) 

1 10.  How  many  teachers  did  Eisenhower  funds  support  to  participate  in  immersion  activities  from  July  1 through  December  1997? 

111.  How  long  did  the  typical  immersion  activity  last? 

a.  One  day  or  less 

b.  More  than  one  day  but  less  than  one  week 

c.  Between  one  week  and  one  month,  or 

d.  Longer  than  one  month 


Mentoring,  coaching,  or  observation 

114.  Did  your  district  support  mentoring,  coaching,  or  observation  activities  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  122) 

115.  Were  any  mentoring/coaching/observing  activities  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  122) 

1 1 8.  How  many  teachers  were  mentored,  coached,  or  observed  from  July  1 through  December  1 997  using  Eisenhower  funds? 

121 .  How  long  did  the  typical  mentor-  or  coach-teacher  relationship  last? 

a.  One  day  or  less 

b.  More  than  one  day  but  less  than  one  week 

c.  More  than  one  week  but  less  than  one  month 

d.  Between  one  month  and  one  year,  or 
d.  More  than  one  year 


Teacher  resource  center 

1 22.  Did  your  district  support  a teacher  resource  center  from  July  1 through  December  1 977? 

1 Yes 

2 No  (skip  to  127) 

123.  Did  you  use  Eisenhower  funds  to  support  the  teacher  resource  center? 

1 Yes 

2 No  (skip  to  127) 

124.  About  how  many  teachers  used  the  teacher  resource  center  from  July  1 through  December  1997? 


EXHIBIT  G.5.1  Id  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


Teacher  committees  or  task  forces 

127.  Did  your  district  support  teacher  committees  or  task  forces  to  improve  teaching  and  learning  from  July  1 through  December 
1977? 

1 Yes 

2 No  (skip  to  135) 

128.  Were  any  teacher  committees  or  task  forces  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  135) 

130.  How  many  such  groups  did  Eisenhower  funds  support  from  July  1 through  December  1997? 

131 . About  how  many  teachers  participated  in  the  typical  committee  or  task  force? 

134.  Over  what  period  of  time  did  the  typical  task  force  or  committee  meet? 

a.  One  day  or  less 

b.  More  than  one  day  but  less  than  one  week 

c.  More  than  one  week  but  less  than  one  month 

d.  Between  one  month  and  one  year,  or 
d.  More  than  one  year 


Teacher  study  groups 

135.  Did  your  district  support  teacher  study  groups  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  142) 

136.  Were  any  teacher  study  groups  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  142) 

138.  How  many  such  groups  did  Eisenhower  funds  support  from  July  1 through  December  1997? 

139.  About  how  many  teachers  participated  in  the  typical  study  group? 

141.  Over  what  period  of  time  did  the  typical  study  group  meet? 

a.  One  day  or  less 

b.  More  than  one  day  but  less  than  one  week 

c.  More  than  one  week  but  less  than  one  month 

d.  Between  one  month  and  one  year,  or 
d.  More  than  one  year 
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Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


Individual  research  projects 

142.  Did  your  district  support  individual  research  projects  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  147) 

143.  Were  any  individual  research  projects  supported,  at  least  in  part,  with  Eisenhower  funds? 

1 Yes 

2 No  (skip  to  142) 

145.  How  many  such  projects  did  Eisenhower  funds  support  from  July  1 through  December  1997? 

146.  How  long  did  the  typical  research  project  last? 

a.  One  week  or  less 

b.  More  than  one  week  to  a month 

c.  More  than  a month  to  a year 

d.  More  than  a year 

Other  professional  development 

147.  Did  your  district  support  any  other  professional  development  from  July  1 through  December  1977? 

1 Yes 

2 No  (skip  to  150) 

1 48.  Did  you  use  Eisenhower  funds  to  support  any  of  these  activities? 

1 Yes 

2 No  (skip  to  142) 


Systemic  reform  efforts  and  the  Eisenhower  Program 

154.  Have  STATE-WIDE  standards  or  curriculum  frameworks  in  MATHEMATICS  been  adopted? 

1 Yes 

2 No  (skip  to  156) 

155.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
STATE-WIDE  standards  or  frameworks  in  MATHEMATICS? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  statewide  mathematics  standards  and 
frameworks 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

1 56.  Have  DISTRICT-WIDE  standards  or  curriculum  frameworks  in  MATHEMATICS  been  adopted? 

1 Yes 

2 No  (skip  to  159) 
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Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


157.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
DISTRICT-WIDE  standards  or  frameworks  in  MATHEMATICS? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  district-wide  mathematics  standards  and 
frameworks 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

154.  Have  STATE-WIDE  standards  or  curriculum  frameworks  in  SCIENCE  been  adopted? 

1 Yes 

2 No  (skip  to  1 56) 

155.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
STATE-WIDE  standards  or  frameworks  in  SCIENCE? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  statewide  science  standards  and 
frameworks 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

1 56  Have  DISTRICT-WIDE  standards  or  curriculum  frameworks  in  SCIENCE  been  adopted? 

1 Yes 

2 No  (skip  to  159) 

157.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
DISTRICT-WIDE  standards  or  frameworks  in  SCIENCE? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  district-wide  science  standards  and 
frameworks 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

165.  Are  STATE-WIDE  assessments  in  MATHEMATICS  administered  in  your  district? 

1 Yes 

2 No  (skip  to  167) 

166.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
STATE-WIDE  assessment  requirements  in  MATHEMATICS? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  statewide  mathematics  assessment 
requirements 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

167.  Does  your  district  administer  DISTRICT-WIDE  assessments  in  MATHEMATICS? 

1 Yes 

2 No  (skip  to  170) 
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Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


168.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
DISTRICT-WIDE  assessment  requirements  in  MATHEMATICS? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  district-wide  mathematics  assessment 
requirements 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

170.  Are  STATE-WIDE  assessments  in  SCIENCE  administered  in  your  district? 

1 Yes 

2 No  (skip  to  172) 

166.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
STATE-WIDE  assessment  requirements  in  SCIENCE? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  statewide  science  assessment 
requirements 

b.  The  activities  are  designed  to  some  extent  to  do  this,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 

167.  Does  your  district  administer  DISTRICT-WIDE  assessments  in  SCIENCE? 

1 Yes 

2 No  (skip  to  170) 

168.  To  what  extent  are  Eisenhower-supported  activities  in  your  district  designed  to  help  teachers  adapt  their  teaching  to  meet 
DISTRICT-WIDE  assessment  requirements  in  SCIENCE? 

a.  The  activities  are  not  at  all  designed  to  help  teachers  adapt  their  teaching  to  these  district-wide  science  assessment 
requirements 

b.  The  activities  are  designed  to  some  extent  to  do  thjs,  or 

c.  The  activities  are  designed  to  a large  extent  to  do  this 
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Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 

PART  B 

Now  I’d  like  to  ask  about  various  Federal  programs  that  may  operate  in  your  state  or  district,  and  the  relationship  between  these 
programs  and  the  Eisenhower  program. 

5-61 . Please  answer  the  following  questions  about  these  Federal  programs.  (Items  number  from  5 to  61 , beginning  with  first  row) 
[Interviewer:  Ask  each  question  about  SSI,  then  about  USI,  etc.] 


Did 

[PROGRAM] 
operate  in  your 
state  or  district 
last  year? 

Did  it  support 
professional 
development  in 
your  district  last 
year? 

Did  the  program 
co-fund 
professional 
development  with 
Eisenhower  in 
the  last  year? 

Did  program  staff 
work  closely  with 
Eisenhower  staff  in 
the  last  year? 

National  Science  Foundation: 
Statewide  Systemic  Initiative 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

National  Science  Foundation: 
Urban  Systemic  Initiative 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

National  Science  Foundation: 
Rural  Systemic  Initiative 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

National  Science  Foundation: 
Local  Systemic  Change 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Title  1:  Part  A (Helping 
Disadvantaged  Children) 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Title  1:  Part  C (Education  of  Migrant 
Children) 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Title  III:  Technology  for  Education 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Title  IV:  Innovative  Education 
Program  Strategies 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Title  VII:  Bilingual  Education 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Title  IX:  Indian  Education 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Individuals  with  Disabilities 
Education  Act  (IDEA) 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Goals  2000 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

School  to  work  programs 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Perkins  Vocational  Education  Funds 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 

Other  (please  explain) 

Y N 
Don’t  know 

Y 

N 

Y 

N 

Y 

N 
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EXHIBIT  G.5.11d  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


Targeting  and  recruitment 

65-74:  How  much  emphasis  do  you  give  to  recruiting: 


No 

Particular  Some  Strong 

Emphasis  Emphasis  Emphasis 


a.  Teachers  from  Title  I schools 0 

b.  Title  I teachers 0 

c.  Special  education  teachers 0 

d.  Teachers  of  limited  English  proficiency 

students 0 

e.  Teachers  from  schools  with  low  achievement  levels 

0 


f.  Teachers  from  high  poverty  schools  (50%  or  more  students  eligible 
for  free/reduced  price  lunch) 


0 

g.  Paraprofessionals 0 

h.  Other  (specify) 0 


1 

1 

1 


2 

2 

2 


1 2 


1 2 


1 

1 

1 


2 

2 

2 


75.  In  which  of  the  following  ways  do  you  try  to  increase  the  participation  of  teachers,  paraprofessionals,  or  other  populations?  (circle 
all  that  apply) 

Do  you.... 

a.  Publicize  activities 

b.  Tailor  the  focus  of  professional  development  towards  the  needs  of  special  population 

c.  Use  incentives 

d.  Do  you  use  other  strategies? 

e.  No  special  strategies  are  used 

Processes  and  procedures 

78.  Are  teachers’  needs  for  professional  development  formally  assessed? 

1 Yes 

2 No  (skip  to  82) 


79.  In  which  of  the  following  ways  are  teachers’  needs  for  professional  development  assessed?  (circle  all  that  apply) 

a.  With  a survey  of  teachers 

b.  With  meetings  of  teacher  representatives 

c.  With  a survey  of  principals  or  department  chairs 

d.  With  measures  of  student  performance 

e.  With  informal  conversations 

f.  Are  teachers’  needs  assessed  in  other  ways  (please  specify) 
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EXHIBIT  G.5.1  Id  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


83.  Do  you  evaluate  Eisenhower-supported  professional  development? 

1 Yes 

2 No  (skip  to  85) 


84.  In  which  of  the  following  ways  do  you  evaluate  Eisenhower-supported  professional  development?  (circle  all  that  applyO 
With  or  by.... 

a.  By  number  of  teachers  participating  in  professional  development 

b.  With  a teacher  satisfaction  survey 

c.  With  observations  of  teachers 

d.  With  student  achievement  scores 

e.  In  other  ways  (please  specify) 


88.  Has  your  DISTRICT  developed  performance  indicators  for  professional  development? 

1 Yes 

2 No  (skip  to  90) 

89.  Is  your  DISTRICT  currently  developing  performance  indicators  for  professional  development? 

1 Yes 

2 No  (skip  to  96) 


142.  Do  district  Eisenhower  staff  EXCHANGE  IN  INFORMATION  with  SCHOOLS  regarding  professional  development? 

1 Yes 

2 No  (skip  to  144) 


143.  In  which  of  the  following  ways  do  district  Eisenhower  staff  exchange  information  with  schools  regarding  professional 
development?  (circle  all  that  apply) 

Through.... 

a.  Regular  visits  and  observations 

b.  Telephone  calls  to  schools 

c.  Regular  required  reports  from  schools 

d.  Required  evaluations  of  school-level  professional  development  programs 

e.  In  other  ways  (please  specify) 


1 44.  Do  district  Eisenhower  staff  provide  assistance  to  schools? 

1 Yes 

2 No  (skip  to  146) 
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EXHIBIT  G.5.1  Id  (Continued) 

Items  from  District  Telephone  Interview  Used  in  Analyses  Reported  in  Chapters  4 and  5 


145.  Which  of  the  following  kinds  of  assistance  do  they  provide?  (all  that  apply) 

Do  they.... 

a.  Provide  guidance  in  interpreting  Title  II  rules  and  regulations 

b.  Help  conducting  needs  assessments 

c.  Help  developing  professional  development  plan 

d.  Help  developing  specific  activities 

e.  Help  developing  performance  indicators  for  professional  development 

f.  Assist  in  other  ways  (please  specify) 

146.  Do  district  Eisenhower  staff  EXCHANGE  INFORMATION  with  professional  development  providers? 

1 Yes 

2 No  (skip  to  148) 

147.  In  which  of  the  following  ways  do  they  exchange  information? 

Through.... 

a.  Regular  visits  and  observations  of  professional  development  activities 

b.  Telephone  calls  to  professional  development  providers 

c.  Regular  required  reports  from  providers 

d.  Required  evaluations  of  professional  development  activities 

e.  Other  ways  (please  specify) 

148.  Do  district  Eisenhower  staff  provide  assistance  to  professional  development  providers? 

1 Yes 

2 No  (skip  to  150) 

149.  Which  of  the  following  kinds  of  assistance  do  district  Eisenhower  staff  provide  to  professional  development  providers? 
Do  they 

a.  Provide  guidance  in  interpreting  Title  II  rules  and  regulations 

b.  Help  conducting  needs  assessments 

c.  Help  developing  professional  development  plans 

d.  Help  developing  specific  activities 

e.  Help  developing  performance  indicators  for  professional  development 

f . Assist  in  other  ways  (please  specify) 
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APPENDIX  H 


SUPPLEMENT  FOR  CHAPTER  6 


This  appendix  provides  supplementary  information  for  the  analyses  reported  in  Chapter  6. 
Each  Exhibit  in  the  appendix  corresponds  to  an  Exhibit  of  the  same  number  in  Chapter  6.  The  boxes 
at  the  bottom  of  each  Exhibit  refer  to  SAHE-gran tee  Project  Director  Telephone  Survey  item 
numbers  or  composite  variables  on  which  the  exhibit  is  based.  Eisenhower  Project  Director  Survey 
item  numbers  begin  with  the  letter  “E,”  followed  by  the  section  of  the  survey  (part  A or  B)  and  the 
item  involved.  For  example,  item  numbers  “EA069_1,  EA069_2,  and  EA069_3”  refer  to  the  Project 
Director  Survey  part  A,  item  69_1,  2,  and  3.  (Item  69_1  is  shown  on  printed  copies  of  the  survey  as 
69a,  69_2  is  shown  as  69b,  etc.)  All  items  referred  to  in  the  chapter  appear  in  Exhibit  6.26d. 
Composite  variables,  which  appear  in  all  capital  letters  (for  example,  PDIRT),  are  defined  in  Exhibit 
6.26b. 


EXHIBIT  H.6.2 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  with  Primary  Activities 
that  Cover  Mathematics,  Science,  and  Other  Subject  Areas  (n=92) 


Subject  Areas 

Percent  of  Teachers  Participating 
in  SAHE-grantee  Activities 

Mathematics  Only 

27.1 

Science  Only 

18.5 

Mathematics/Science 

18.3 

Mathematics/Science/Other 

28.2 

Other  Only 

8.0 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables.  EA069_I— EA069.3 

EXHIBIT  H.6.3 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  that  Offer  “Traditional” 

Activities  (n=92) 


Types  of  “Traditional”  Activities Offer  Activity Offer  Activity  as  Primary  Focus 


College  course 
Workshops 

38.0 

76.3 

26.1 

55.3 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables’.  EA067,  EB005,  EB008,  EB01 1 

EXHIBIT  H.6.4 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  that  Offer  “Reform” 

Activities  (n=92) 


Types  of  “Reform”  Activities 

Offer  Activity 

Offer  Activity  as  Primary  Focus 

Study  Group 

25.1 

4.0 

Teacher  Networking 

43.0 

8.9 

Mentoring 

40.6 

1.6 

Committee  or  Task  Force 

15.4 

0.0 

Internship 

6.8 

0.0 

Individual  Research 

11.9 

0.0 

Resource  Center 

32.0 

0.0 

Source.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

Variables'.  Responses  combine  primary  activity  (EA067_5  — EA067_1 1)  and  other  activities  (BOM,  BO  16,  BO  18,  BO 20, 
B022,  B024,  B026)  stud.all,  stud_pri,  net.all,  net_pri,  ment_all,  ment.pri,  comm_all,  comm_pri,  intn_all,  intn.pri, 
proj„all,  proj_pri,  reso_all,  reso_pri 


EXHIBIT  H.6.5 

SAHE-grantees’  Support  for  “Reform”  Types  of  Activities,  Overall  and  by  Institution 

Type  and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

IHE  Overall 

13.85 

(35.00) 

86 

Institutional  Type 

2.72 

1.80 

.10 

Research 

9.88 

(33.77) 

31 

Other 

16.97 

(35.70) 

55 

Dept.  Affiliation 

3.73 

2.80 

.02 

Math/Science  Education  Other 

Math/Science 

1.96 

(13.93) 

42 

Education 

22.09 

(45.16) 

30 

X 

Other 

31.94 

(43.94) 

14 

X 

Inst’Dept 

1.11 

2.80 

.33 

NPO  Overall 

27.62 

(43.37) 

6 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998 
Variables:  NET.REF 
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EXHIBIT  H.6.6a 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  by  Contact 

Hours  of  Primary  Activity  (n=92) 


Number  of  Hours  of  Primary  Activity 

Percent  of  Teachers  Participating 
in  SAHE-grantee  Activities 

Less  than  4 

1.48 

4 to  8 

9.5 

9 to  40 

31.52 

40  or  more 

57.5 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : Amount-(=EA080) 

EXHIBIT  H.6.6b 

Contact  Hours  of  SAHE-grantees’  Primary  Activity,  Overall  and  by  Institution 
Type  and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 
(where  p<.  OS) 

IHE  Overall 

64.11 

95.14 

86 

Institutional  Type 

1.15 

1,80 

.28 

Research 

82.32 

(151.26) 

31 

Other 

49.78 

(33.41) 

55 

Dept.  Affiliation 

3.71 

2,  80 

.02 

Math/Science  Education  Other 

Math/Science 

44.74 

(36.32) 

42 

Education 

98.36 

(148.53) 

30 

Other 

31.53 

(23.94) 

14 

Inst’Dept 

4.45 

2,  80 

.01 

NPO  Overall 

27.37 

(15.48) 

6 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Scale  Definition:  amountj=ea080 
Variables.  EA080 

o 
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EXHIBIT  H.6.6C 


Contact  Hours  of  SAHE-grantees’  Primary  Activity,  Interaction  of  Institution  Type  and 

Departmental  Affiliation  (n=86) 


Departmental  Affiliation 

Research/Doctoral 

SD 

Other 

SD 

Math/Science 

26.45 

(29.71) 

58.36 

(34.28) 

Education 

140.37 

(198.81) 

56.19 

(34.71) 

Other 

49.48 

(35.14) 

26.77 

(17-51) 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Scale  Definition : amount_i=ea080 
Variables.  EA080 


EXHIBIT  H.6.7a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  by  Time 
Span  of  Primary  Activity  (n=92) 


Span  of  Primary  Activities 

Percent  of  Teachers  Participating 
in  SAHE-grantee  Activities 

1 day 

6.94 

2-7  days 

7.69 

8 days  to  1 month 

6.54 

1 month  to  1 year 

52.27 

>1  year 

26.57 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables:  EA077  (#  span_i) 
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EXHIBIT  H.6.7b 


Span  of  SAHE-grantees’  Primary  Activity,  Overall  and  by  Institution  Type 
and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F df 

P 

Significant  Tukey  Pairwise  Contrasts 
(where  p<. 05) 

IHE  Overall 

3.83 

(1.14) 

86 

Institutional  Type 

0.83  1,80 

0.37 

Research 

3.96 

(1.40) 

31 

Other 

3.73 

(0.97) 

55 

Dept.  Affiliation 

7.24  2, 80 

0.00 

Math/Science  Education  Other 

Math/Science 

3.38 

(1.43) 

42 

Education 

4.31 

(0.55) 

30 

X 

Other 

4.04 

(0.38) 

14 

Inst’Dept 

.84  2,80 

.43 

NPO  Overall 

3.88 

(0.53) 

6 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables.  EA077  (=span_i) 

EXHIBIT  H.6.8a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  Whose  Primary  Activity 

Involves  Collective  Participation  (n=92) 

Types  of  Participants  Percent 

All  teachers  in  a school  1 4.34 

All  teachers  in  department  or  grade 15.24 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

Variables:  EA097_3,  EA097_4 
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EXHIBIT  H.6.8b 


Collective  Participation  in  SAHE-grantee  Projects,  Overall  and  by  Institution  Type  and 

Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

IHE  Overall 

0.15 

(0.34) 

86 

Institutional  Type 

8.09 

1,80 

.00 

Research 

0.24 

(0.40) 

31 

Other 

0.08 

(0.23) 

55 

Dept.  Affiliation 

3.21 

2,  80 

.04 

Math/Science  Education  Other 

Math/Science 

0.12 

(0.32) 

42 

Education 

0.14 

(0.34) 

30 

Other 

0.31 

(0.42) 

14 

Inst’Dept 

0.90 

2,  80 

.41 

NPO  Overall 

0.00 

(0.00) 

6 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Scale  Definition : SCH_BS_I=  mean(EA097_3,  EA097_4) 

Scale  Reliability  : .76 
Variables:  SCH_BS_I 

EXHIBIT  H.6.9a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  Whose  Primary  Activity 

Focuses  on  Content  Knowledge  (n=92) 


Extent  of  Content  Knowledge  Focus Percent 


No  emphasis 

5.30 

Some  emphasis 

23.02 

Strong  emphasis 

71.68 

Source4.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables.  content_I 
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EXHIBIT  H.6.9b 


Extent  of  Content  Knowledge  Focus  in  SAHE-grantees’  Primary  Activity,  Overall  and 
by  Institution  Type  and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

IHE  Overall 

2.67 

(0.56) 

86 

Institutional  Type 

3.28 

1,80 

.07 

Research 

2.78 

(0.64) 

31 

Other 

2.58 

(0.50) 

55 

Dept.  Affiliation 

.58 

2,  80 

.56 

Math/Science  Education  Other 

Math/Science 

2.74 

(0.61) 

42 

Education 

2.62 

(0.52) 

30 

Other 

2.52 

(0.47) 

14 

Inst'Dept 

5.55 

2,  80 

.00 

NPO  Overall 

2.52 

(0.80) 

6 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables'.  Content  I 


EXHIBIT  H.6.9C 

Extent  of  Content  Knowledge  Focus  in  SAHE-grantees’  Primary  Activity,  by 
Institution  Type  and  Departmental  Affiliation  (n=86) 


Departmental  Affiliation 

Research/Doctoral 

Other 

Math/Science 

2.63  (.95) 

2.83  (.42) 

Education 

2.91  (.32) 

2.29  (.48) 

Other 

2.84  (.32) 

2.44  (.50) 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : EA092 
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EXHIBIT  H.6.10a 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  that  Provide  Each  of 
Sixteen  Types  of  Opportunities  for  Active  Learning  in  Primary  Activity  (n=92) 


Types  of  active  learning 

Percent 

Score  Assessments 

22.03 

Participant  Leads  Whole  Group  Discussion 

45.35 

Participants  Observe  Others  Teaching 

46.56 

Review  Student  Work 

47.99 

Participants  Simulate  Practice 

56.50 

Participants  Receive  Coaching 

56.56 

Leader  Observes  Participants  Teaching 

61.45 

Participant  Develops  Curriculum 

63.44 

Participant  Gives  Lecture 

65.04 

Participants  Write  Paper 

67.59 

Informal  Meetings 

69.57 

Participant  Leads  Small  Discussion 

70.74 

Participant  Conducts  Demonstration 

70.82 

Communication  with  Teacher 

78.91 

Participants  Discuss  Implementation 

81.64 

Participants  Share  Work 

82.68 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables ; EA075J-9,  EA108_5,  6, 7,  8,  10,  12,  and  13 
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EXHIBIT  H.6.1  Ob 


Number  of  Types  of  Opportunities  for  Active  Learning  in  SAHE-grantees’ 
Primary  Activity,  Overall  and  by  Institution  Type  and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F df  p 

Significant  Tukey  Pairwise  Contrasts 
(where  p < .05) 

IHE  Overall 

9.8 

(3.49) 

86 

Institutional  Type 

.09  1,80  0.77 

Research 

9.71 

(4.26) 

31 

Other 

9.94 

(3.00) 

55 

Dept.  Affiliation 

5.70  2, 80  .00 

Math/Science  Education  Other 

Math/Science 

8.64 

(3.62) 

42 

Education 

10.8 

(3.35) 

30 

Significant  interaction  effects 

Other 

11.23 

(2.08) 

14 

Inst’Dept 

7.28  2, 80  .00 

NPO  Overall 

10.32 

(4.55) 

6 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

Scale  Definition : ACTIVE_I=  sum(EA075_l,EA075_2,EA075_3,EA075_4,EA075_5,EA075_6,EA075_7,EA075_8,  EA075_9, 
EA 1 08_5,EA  1 08_6,EA  108_7,  EA1 08_8,  EA108_J0,  EA108_12,  EA108_13) 

Scale  Reliability  . .11 
Variables’.  ACTIVE^! 

EXHIBIT  H.6.1  Oc 

Number  of  Types  of  Opportunities  for  Active  Learning  in  SAHE-grantees’  Primary 
Activity,  Interaction  of  Institution  Type  and  Departmental  Affiliation  (n=86) 


Departmental  Affiliation 

Research/Doctoral 

SD 

Other 

SD 

Math/Science 

7.1 

(4.55) 

9.8 

(2.85) 

Education 

12.1 

(2.32) 

9.3 

(3.61) 

Other 

10.5 

(2.09) 

11.4 

(2.15) 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables.  ACTIVE_I 
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EXHIBIT  H.6.11 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  Whose  Directors  Report 
Placing  No,  Some,  or  a Strong  Emphasis  on  Recruiting  Teachers  of  Special  Student 

Populations  (n=92) 


Teachers  by  Type  of  School  or 
Type  of  Students 

Strong  Emphasis 

Some  Emphasis 

No  Emphasis 

Title  1 Schools 

18.37 

36.74 

44.89 

Special  Education 

3.84 

27.29 

68.87 

Limited  English 

3.59 

24.73 

71.67 

Low  Achievement 

41.69 

28.55 

29.76 

High  Poverty 

41.72 

29.74 

28.54 

Source.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables:  EB036  EB037  EB038  EB039  EB040 


EXHIBIT  H.6.12 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  According  to  How 

Teachers  Come  to  Participate  (n=92) 


Types  of  Participation 

Percent 

Volunteer 

78.43 

Principal  selected 

20.05 

Provider  selected 

1.55 

Rotation 

,34 

Other 

.42 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables:  EB031.1  - EB031_5 
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EXHIBIT  H.6.13 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Use  Various 
Strategies  to  Increase  Participation  (n=92) 


Strategies  Percent 


Publicize  Activities 

82.62 

Tailor  Focus 

50.40 

Use  Incentives 

50.82 

Other 

18.11 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : EB043J  - EB043,4 


EXHIBIT  H.6.14a 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  in  which  State  and 
District  Standards  and  Assessments  Play  a Role  in  Project  Design  (n=92) 


Types  of  “Reform”  Activities  Standards  Play  a Role  Assessments  Play  a Role 

State  92.21  57.31 

District 61.64 39.80 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

Variables:  EA032,  EA033,  EA034,  EA03S 
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EXHIBIT  H.6.1 4b 


Degree  of  Alignment  between  SAHE-grantees’  Eisenhower  Project  and  State  and 
District  Standards  and  Assessments,  Overall  and  by  Institution  Type  and 

Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

IHE  Overall 

2.50 

(1.32) 

86 

Institutional  Type 

7.66 

1,80 

.00 

Research 

2.05 

(1.57) 

31 

Other 

2.86 

(1.06) 

55 

Dept.  Affiliation 

1.52 

2, 80 

.22 

Math/Science  Education  Other 

Math/Science 

2.29 

(1.42) 

42 

Education 

2.69 

(1.28) 

30 

Other 

2.72 

(1.09) 

14 

lnst*Dept 

2.08 

2,  80 

.13 

NPO  Overall 

2.56 

(1.14) 

6 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Scale  Definition:  Align_i  = sum(EA032,EA033,EA034,EA035) 

Scale  Reliability : .59 

Variables:  ALIGN_I 


EXHIBIT  H.6.1 5 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  have  Ongoing 
Feedback  Mechanisms  with  Districts  (n=92) 


Feedback  Mechanisms 

Percent 

Regularly  Required  Reports 

14.17 

Professional  Development  Evaluation 

17.96 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

Variables  : EB059.3,  EB059_4 
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EXHIBIT  H.6.16 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Support  and 
Coordinate  with  District  Professional  Development  Activities  and  Programs  (n=92) 


Type  of  SAHE-grantee  Coordination  with  Districts 

Percent 

Support  or  Extend  District  Activities 

73.76 

Coordinate  with  District  Programs 

24,24 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : EA025,  EA062 

EXHIBIT  H.6.17 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Work  with  Districts 

in  Different  Ways  (n=92) 


Ways  SAHE  Grantees  Work  with  Districts 

Percent 

Participate  in  District  Planning 

52.95 

Communicate  with  Staff 

90.07 

Use  District  Needs  Assessment 

70.80 

Regular  District  Visits 

37.66 

Other  Ways 

64.46 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : EA024J—  EA024_4,  EB059J 


EXHIBIT  H.6.18 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  in  Which  District  Staff 
Are  Involved  in  Planning,  Implementing,  or  Monitoring  Eisenhower-assisted  Activities 

(n=92) 


Types  of  People  With  Whom  SAHE  Grantees  Work 

Percent 

District  Eisenhower  Coordinators 

23.84 

Other  District  Administrators 

52.64 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables:  peopinvi  = EA030  IN,  EA030_2N 
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EXHIBIT  H.6.1 9 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Co-fund  and/or 
Work  Closely  with  Other  Federal  Programs  (When  the  Program  Operates  in  the 

District)  (n=92) 


Federal  Programs 

Operates  in 
state/district 

Supports 

professional 

development 

Co-funds  with 

Works  cl< 
with 

SSI 

28.2 

90.7 

27.2 

45.9 

USI 

7.0 

80.6 

15.0 

61.5 

RSI 

3.4 

90.0 

0 

80.9 

LSC 

3.5 

100.0 

61.4 

76.0 

Title  1,  Part  A 

63.5 

83.3 

37.4 

22.4 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 

Variables : work  closely;  EA040  EAQ44  EA048  EA052  EA056;  Co-fund:  EA039  EA043  EA047 


EXHIBIT  H.6.20 

Number  of  Types  of  Coordination  of  SAHE-grantee  Projects  with  Districts,  Overall 
and  by  Institution  Type  and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F df 

P 

Significant  Tukey  Pairwise  Contrasts 

(where  p<. 05) 

IHE  Overall 

5.52 

(2.44) 

86 

- 

Institutional  Type 

2.80  1,80 

.09 

Research 

5.03 

(3.32) 

31 

Other 

5.90 

(1.73) 

55 

Dept.  Affiliation 

6.74  2, 80 

0.00 

Math/Science  Education  Other 

Math/Science 

4.82 

(2.30) 

42 

Education 

6.51 

(2.54) 

30 

X 

Other 

5.07 

(1.90) 

14 

Inst'Dept 

2,  80 

NPO  Overall 

5.50 

(1.84) 

6 

- 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : COORD I 
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EXHIBIT  H.6.21 


Percent  of  Teachers  Participating  in  SAHE-grantee  Projects,  by  Status  and  Effect  of 
District  and  State  Performance  Indicators  on  the  Project  (n=92) 


Status  of  State  and  District  Performance  Indicators 

Percent 

State  Has  Performance  Indicators 

49.6 

Given  State  Indicators,  Effect  on  Eisenhower 

76.5 

District  Developed  at  Least  Some  Performance  Indicators 

39.6 

Given  District  Indicators,  Effect  on  Eisenhower 

21.5 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : EB049  EB05 1 EB052  EB055 

EXHIBIT  H.6.22 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Use  Different 
Strategies  to  Assess  Teachers’  Professional  Development  Needs  (n=92) 


Needs  Assessment  Methods  Percent 


Teacher  Survey 

50.7 

Teacher  Representative  Meeting 

40.7 

Principal  Survey 

19.2 

Measure  Student  Performance 

23.6 

Informal  Conversation 

50.0 

Source:  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables:  EB046  1 - EB046  5 

EXHIBIT  H.6.23 

Percent  of  Teachers  Participating  in  SAHE-grantee  Projects  That  Use  Different 
Methods  for  Evaluating  Activities  (n=92) 


Evaluation  Methods 

Percent 

Teacher  Participation 

61.1 

Teacher  Survey 

91.9 

Observe  Teachers 

59.3 

Student  Achievement 

30.6 

Source'.  Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables : EB048J  - EB048_4 
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EXHIBIT  H.6.24 


Extent  of  SAHE-grantee  Continuous  Improvement  Efforts,  Overall  and  by  Institution 

Type  and  Departmental  Affiliation  (n=92) 


Mean 

SD 

n 

F 

df 

P 

Significant  Tukey  Pairwise  Contrasts 
(where  p < .05) 

IHE  Overall 

1.38 

(0.80) 

86 

Institutional  Type 

8.79 

1,80 

0.00 

Research 

1.17 

(0.87) 

31 

Other 

1.55 

(0.73) 

55 

Dept.  Affiliation 

20.77 

2,80 

0.00 

Math/Science  Education  Other 

Math/Science 

0.91 

(0.50) 

42 

Education 

1.83 

(0.73) 

30 

X 

Other 

1.71 

(0.93) 

14 

X 

lnst*Dept 

.88 

2,  80 

.42 

NPO  Overall 

1.76 

(0.64) 

6 

Source : Telephone  Survey  of  SAHE-grantee  Project  Directors,  Spring  1998. 
Variables'.  CONIMP_I 
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EXHIBIT  H.6.25d 


Items  from  SAHE-Grantee  Project  Director  Telephone  Interview  Used  in  Analyses 

Reported  in  Chapter  6 

SAHE-GRANTEE  PROJECT  DIRECTOR  TELEPHONE  INTERVIEW 

NOTE:  The  IHE/NPO  Project  Director  Interview  was  administered  as  a Computer-Assisted  form,  and  the  items  as  administered 
appeared  on  a computer  screen.  The  following  printed  items  parallel  the  versions  that  appeared  on  the  CATI  screen,  but  some  details 
have  been  changed. 

PART  A 


Eisenhower  project 

5.  Which  best  describes  your  organization.  Is  it  an  institution  of  higher  education  or  another  not-for-profit  organization? 

1.  Institution  for  higher  education 

2.  Other  not-for-profit  organization 

6.  What  type  of  college  or  university  do  you  work  in?  Do  you  work  in: 

1 . A research  institution 

2.  A doctoral  granting  institution 

3.  An  institution  that  primarily  grants  bachelor's  and  master’s  degrees 

4.  A private  liberal  arts  institution 

5.  A public  two-year  institution 

6.  Another  type  of  institution.  Please  specify 

7.  Which  department,  school,  or  center  are  you  affiliated  with  at  your  institution?  Are  you  affiliated  with: 

1 . The  math  department 

2.  The  science  department 

3.  The  education  school  or  department 

4.  A research  center 

5.  Another  department  or  school 

24.  How  do  you  work  with  the  district?  Do  you: 

a.  Participate  in  district  planning 

b.  Communicate  periodically  with  district  staff 

c.  Rely  on  district  needs  assessments  to  plan  your  project 

d.  Work  with  the  district  in  other  ways 

e.  Or  do  you  not  work  very  closely  with  the  district 

25.  Is  your  professional  development  activity  designed  to  support  or  extend  other  professional  development  activities  in  the  district? 

1.  Yes 

2.  No 

32.  Did  state  standards  and  frameworks  play  a role  in  designing  your  project? 

1 . Yes.  If  yes,  how  did  they  play  a role? 

2.  No  - 

33.  Did  district  standards  and  frameworks  play  a role  in  designing  your  project? 

1 . Yes.  If  yes,  how  did  they  play  a role? 

2.  No 
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EXHIBIT  H.6.25d  (Continued) 


Items  from  SAHE-grantee  Project  Director  Telephone  Interview  Used  in  Analyses 

Reported  in  Chapter  6 


34.  Did  state  assessments  play  a role  in  designing  your  project? 

1 . Yes.  If  yes,  how  did  they  play  a role? 

2.  No 

35.  Did  district  assessments  play  a role  in  designing  your  project? 

1 . Yes.  If  yes,  how  did  they  play  a role? 

2.  No 

37.  Did  a National  Science  Foundation  State  Systemic  Initiative  operate  in  the  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  41) 

3.  Don’t  know  (skip  to  41) 

38.  Did  the  program  support  professional  development  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  40) 

39.  Did  the  program  co-fund  professional  development  last  year  with  your  Eisenhower  project? 

1.  Yes 

2.  No 

40.  Did  program  staff  work  closely  with  your  Eisenhower  project  staff  in  the  last  year? 

1.  Yes 

2.  No 

41.  Did  a National  Science  Foundation  Urban  Systemic  Initiative  operate  in  the  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  45) 

3.  Don’t  know  (skip  to  45) 

42.  Did  the  program  support  professional  development  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  44) 

43.  Did  the  program  co-fund  professional  development  last  year  with  your  Eisenhower  project? 

1.  Yes 

2.  No 

44.  Did  program  staff  work  closely  with  your  Eisenhower  project  staff  in  the  last  year? 

1.  Yes 

2.  No 

45.  Did  a National  Science  Foundation  Rural  Systemic  Initiative  operate  in  the  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  49) 

3.  Don’t  know  (skip  to  49) 
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46.  Did  the  program  support  professional  development  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  48) 

47.  Did  the  program  co-fund  professional  development  last  year  with  your  Eisenhower  project? 

1.  Yes 

2.  No 

48.  Did  program  staff  work  closely  with  your  Eisenhower  project  staff  in  the  last  year? 

1.  Yes 

2.  No 

49.  Did  a National  Science  Foundation  Local  Systemic  Initiative  operate  in  the  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  53) 

3.  Don’t  know  (skip  to  53) 

50.  Did  the  program  support  professional  development  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  52) 

51 . Did  the  program  co-fund  professional  development  last  year  with  your  Eisenhower  project? 

1.  Yes 

2.  No 

52.  Did  program  staff  work  closely  with  your  Eisenhower  project  staff  in  the  last  year? 

1.  Yes 

2.  No 

53.  Did  Title  I,  Part  A (Helping  Disadvantaged  Children)  operate  in  the  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  57) 

3.  Don’t  know  (skip  to  57) 

54.  Did  the  program  support  professional  development  last  year  in  the  districts  with  which  you  work? 

1.  Yes 

2.  No  (skip  to  56) 

55.  Did  the  program  co-fund  professional  development  last  year  with  your  Eisenhower  project? 

1.  Yes 

2.  No 

56.  Did  program  staff  work  closely  with  your  Eisenhower  project  staff  in  the  last  year? 

1 . Yes 

2.  No 
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CHARACTERISTICS  OF  EISENHOWER-SUPPORTED  ACTIVITIES  - PRIMARY  ACTIVITY 

I would  like  now  to  ask  about  the  characteristics  of  your  Eisenhower-supported  activities. 

For  our  purposes,  we  have  defined  a single  activity  as  an  event  with  a common  group  of  participants.  For  example,  if  four  different 
groups  of  teachers  attended  the  same  workshop  on  four  separate  occasions,  this  would  count  as  four  activities.  But  if  one  group  of 
teachers  attended  a workshop  and  a follow-up  event,  this  would  count  as  one  activity. 

64.  Using  this  definition,  how  many  Eisenhower-supported  professional  development  activities  did  you  lead  from  Julyl  through 
December  1997. 

In  the  following  questions,  we  will  ask  you  to  describe  a professional  development  activity  you  provided  using  Eisenhower  funds.  If 
you  provided  more  than  one  Eisenhower-supported  activity,  please  pick  your  primary  activity. 

66.  What  is  the  name  of  your  primary  activity? 

67.  Which  of  the  following  best  describes  the  activity?  Choose  only  one  response.  If  more  than  one  response  fits  your  activity, 
pick  the  response  that  best  describes  the  aspect  of  the  activity  in  which  participants  spend  the  most  time. 

a.  In-district  workshop  or  institute 

b.  Out-of-district  workshop  or  institute 

c.  Out-of-district  conference 

d.  College  course 

e.  Teacher  collaborative  or  network 

f.  Internship  or  immersion  activity 

g.  Mentoring,  coaching,  observation 

h.  Teacher  resource  center 

i.  Teacher  committee  or  task  force 

j.  Teacher  study  group 

k.  Individual  research  project 

l.  Other  (please  specify) 

69.  On  what  subject  areas  did  the  activity  focus?  Did  it  focus  on 

a.  Mathematics 

b.  Science 

c.  Other  subjects  (specify) 

74.  As  part  of  the  professional  development  activity,  including  preliminary  and  follow-up  sessions,  did  participants  have  the 
opportunity  to  try  out  what  they  learned  in  their  classroom  and  obtain  feedback? 


1.  Yes 

2.  No  (skip  to  77) 
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75.  How  did  this  activity  help  participants  use  new  skills  in  their  classroom?  (Circle  all  that  apply.) 

a.  Participants  practiced  under  simulated  conditions,  with  feedback 

b.  Participants  received  coaching  or  mentoring  in  the  classroom 

c.  Participants  met  formally  with  other  activity  participants  to  discuss  classroom  implementation 

d.  You  or  other  activity  leaders  observed  participants’  teaching  and  provide  feedback 

e.  Participants’  teaching  was  observed  by  other  participants  and  feedback  was  provided 

f.  Participants  communicated  with  you  concerning  classroom  implementation 

g.  Participants  shared  their  students’  work  which  you  or  other  participants  reviewed 

h.  Participants  met  informally  with  other  participants  to  discuss  classroom  implementation 

i.  Participants  developed  curricula  or  lesson  plans,  which  you  or  other  participants  reviewed 

j.  Other  (specify) 

77.  Over  what  period  of  time  was  the  activity  spread,  including  the  main  activity  and  any  formal  preliminary  or  follow-up  sessions? 

1 . One  day  or  less 

2.  More  than  one  day  but  less  than  one  week 

3.  More  than  one  week  but  less  than  one  month 

4.  More  than  one  month  but  less  than  one  year,  or 

5.  More  than  one  year 

80.  How  many  hours  did  the  typical  participant  engage  in  as  part  of  this  activity?  Include  the  main  activity  as  well  as  any  preliminary 
or  formal  follow-up  sessions. 

92.  How  much  emphasis  did  the  activity  give  to  deepening  content  knowledge? 

1.  No  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 

97.  Which  of  the  following  groups  participated  in  the  activity?  Did...  (Circle  all  that  apply) 

a.  Teachers  as  individuals 

b.  Teachers  as  representatives  of  their  department,  grade  level,  or  school 

c.  All  teachers  in  department  or  grade-level  groupings 

d.  All  teachers  in  a school  or  set  of  schools 

e.  Other  configurations  (specify) 
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108.  Which  of  the  following  did  participants  engage  in  during  the  activity?  Did  they...  (Circle  all  that  apply) 

a.  Listen  to  a lecture 

b.  Observe  a demonstration  of  a lesson  or  unit 

c.  Participate  in  a whole-group  discussion 

d.  Participate  in  a small-group  discussion 

e.  Give  a lecture  or  presentation 

f.  Conduct  a demonstration  of  a lesson  or  unit 

g.  Lead  a whole-group  discussion 

h.  Lead  a small-group  discussion 

i.  Engage  in  extended  problem  solving 

j.  Write  a paper,  report  or  plan 

k.  Develop  or  review  materials 

l.  Reviewed  student  work 

m.  , Score  assessments 

n.  Collaborate  as  a colleague  with  mathematicians  or  scientists 

o.  Use  technology  (computers,  calculators,  or  the  internet) 

p.  Complete  paper-and-pencil  problems  or  exercises 

q.  Assess  other  participants’  knowledge  or  skills 

r.  Engage  in  anything  else?  What  else  did  participants  engage  in  during  the  activity? 

PARTB 

OTHER  EISENHOWER-ASSISTED  ACTIVITIES 

4.  Other  than  the  activity  you  have  been  describing,  did  you  support  any  other  activities  over  the  period  from  July  1 , 1997  through 

December  31  using  Eisenhower  funds? 

1.  Yes 

2.  No  (skip  to  30) 

5.  Did  you  support  in-district  workshops  or  institutes  (that  is,  workshops  intended  for  teachers  from  a single  district?) 

1.  Yes 

2.  No  (skip  to  8) 

7.  About  how  many  teachers  participated  in  these  in-district  workshops  or  institutes? 

8.  Did  you  support  out-of-district  workshops  or  conferences  (that  is,  workshops  intended  for  teachers  from  a multiple  districts?) 

1.  Yes  ' 

2.  No  (skip  to  11) 

10.  About  how  many  teachers  participated  in  these  out-of-district  workshops  or  conferences? 

11.  Did  you  support  college  courses  for  credit? 

1.  Yes 

2.  No  (skip  to  14) 

13.  About  how  many  teachers  participated  in  these  college  courses? 
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14.  Did  you  support  teacher  collaboratives  or  networks? 

1.  Yes 

2.  No  (skip  to  16) 

1 5.  About  how  many  teachers  participated  in  these  teacher  collaboratives  or  networks? 

16.  Did  you  support  internships  or  immersion  activities? 

1.  Yes 

2.  No  (skip  to  18) 

17.  About  how  many  teachers  participated  in  these  internship  or  immersion  activities? 

18.  Did  you  support  mentoring,  coaching.'or  observation? 

1.  Yes 

2.  No  (skip  to  20) 

19.  About  how  many  teachers  participated  in  these  mentoring,  coaching,  or  observation  activities? 

20.  Did  you  support  a teacher  resource  center? 

1.  Yes 

2.  No  (skip  to  22)  , 

21 . About  how  many  teachers  used  this  resource  center? 

22.  Did  you  support  teacher  communities  or  task  forces? 

1.  Yes 

2.  No  (skip  to  24) 

23.  About  how  many  teachers  participated  in  these  committees  or  task  forces? 

24.  Did  you  support  teacher  study  groups? 

1.  Yes 

2.  No  (skip  to  26) 

25.  About  how  many  teachers  participated  in  these  teacher  study  groups? 

26.  Did  you  support  teacher  or  other  school  staff  in  conducting  individual  research  projects? 

1.  Yes 

2.  No  (skip  to  28) 

27.  About  how  many  teachers  or  other  school  staff  participated  in  these  individual  research  projects? 

28.  Did  you  support  any  other  activities? 

1.  Yes 

2.  No  (skip  to  30) 

29.  About  how  many  teachers  participated  in  these  other  activities? 
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Targeting  and  recruitment 

31.  Of  the  teachers  in  your  Eisenhower  activities,  what  percent  come  to  participate  in  each  of  the  following  ways? 
a What  percent  volunteer? 

b What  percent  are  selected  by  their  principal  or  other  administrator? 
c What  percent  participate  by  rotation? 
d What  percent  come  to  participate  in  other  ways? 

35.  Do  you  try  to  increase  the  participation  of  either  teachers  from  historically  under  represented  populations  or  teachers  from  specific 
types  of  schools  (for  example,  teachers  of  low-income,  limited  English  proficiency,  or  racial/ethnic  minority  students)? 

36.  How  much  emphasis  do  you  give  to  recruiting  Title  I teachers  for  your  activities? 

1.  No  particular  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 

37.  How  much  emphasis  do  you  give  to  recruiting  special  education  teachers  for  your  activities? 

1.  No  particular  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 

38.  How  much  emphasis  do  you  give  to  recruiting  teachers  of  limited  English  proficiency  students  for  your  activities? 

1.  No  particular  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 

39.  How  much  emphasis  do  you  give  to  recruiting  teachers  from  schools  with  low  achievement  levels  for  your  activities? 

1.  No  particular  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 

40.  How  much  emphasis  do  you  give  to  recruiting  teachers  from  high  poverty  schools  (50  percent  or  more  students  eligible  for  free 
and  reduced  lunch)  for  your  activities? 

1.  No  particular  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 

41.  How  much  emphasis  do  you  give  to  recruiting  paraprofessionals  for  your  activities? 

1.  No  particular  emphasis 

2.  Some  emphasis,  or 

3.  Strong  emphasis 


H-27 


EXHIBIT  H.6.25d  (Continued) 


Items  from  SAHE-grantee  Project  Director  Telephone  Interview  Used  in  Analyses 

Reported  in  Chapter  6 

43.  In  which  of  the  following  ways  do  you  try  to  increase  the  participation  of  teachers,  paraprofessionals,  or  other  populations? 
(Circle  all  that  apply) 

Do  you.... 

a.  Publicize  activities 

c.  Use  incentives 

c.  Tailor  the  focus  of  professional  development  towards  the  needs  of  special  populations 

d.  Do  you  use  other  strategies? 

Processes  and  procedures 

46.  In  which  of  the  following  ways  are  teachers’  needs  for  professional  development  assessed?  (Circle  all  that  apply) 

a.  With  a survey  of  teachers 

b.  With  meetings  of  teacher  representatives 

c.  With  a survey  of  principals  or  department  chairs 

d.  With  measures  of  student  performance 

e.  With  informal  conversations 

f.  Are  teachers'  needs  assessed  in  other  ways  (please  specify) 

47.  Do  you  evaluate  Eisenhower-supported  professional  development? 

1 Yes 

2 No  (skip  to  49) 

48.  In  which  of  the  following  ways  do  you  evaluate  Eisenhower-supported  professional  development?  (Circle  all  that  apply) 

a.  By  number  of  teachers  participating  in  professional  development 

b.  With  a teacher  satisfaction  survey 

c.  With  observations  of  teachers 

d.  With  student  achievement  scores 

e.  In  other  ways  (please  specify) 

49.  Are  you  aware  of  any  performance  indicators  for  professional  development  set  by  the  state? 

1 . Yes 

2.  No  (skip  to  52) 

51 . Have  they  affected  your  Eisenhower  project? 

1 . Yes.  How  have  they  affected  your  Eisenhower  project? 

2.  No 

52.  Have  the  districts  you  work  with  developed  performance  indicators  for  professional  development? 

1 . All  have  (skip  to  54) 

2.  Most  have  (skip  to  54) 

3.  Some  have  (skip  to  54) 

4.  None  have 

5.  Don’t  know 
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53.  Are  the  districts  you  are  working  with  developing  performance  indicators  for  professional  development? 

1.  All  are  (skip  to  56) 

2.  Most  are  (skip  to  56) 

3.  Some  are  (skip  to  56) 

4.  None  are  (skip  to  56) 

5.  Don’t  know  (skip  to  56) 

55.  Have  they  affected  your  Eisenhower  project? 

1 . Yes.  How  have  they  affected  your  Eisenhower  project? 

2.  No 

59.  Do  any  of  the  following  ongoing  feedback  mechanisms  exist  between  you  and  the  district? 

a.  Regular  visits  and  observations  of  professional  development  activities  by  the  district 

b.  Telephone  calls 

c.  Regular  required  reports 

d.  Required  evaluations  of  professional  development  activities 

e.  Other  feedback  mechanisms 

f.  No  ongoing  feedback 
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